extracting low dimensional latent structure from time series in the presence of delays noisy high dimensional time series observations can often be described by a set of low dimensional latent variables commonly used methods to extract these latent variables typically assume instantaneous relationships between the latent and observed variables in many physical systems changes in the latent variables manifest as changes in the observed variables after time delays techniques that do not account for these delays can recover a larger number of latent variables than are present in the system thereby making the latent representation more difficult to interpret in this work we introduce a novel probabilistic technique time delay gaussian process factor analysis td gpfa that performs dimensionality reduction in the presence of a different time delay between each pair of latent and observed variables we demonstrate how using a gaussian process to model the evolution of each latent variable allows us to tractably learn these delays over a continuous domain additionally we show how td gpfa combines temporal smoothing and dimensionality reduction into a common probabilistic framework we present an expectation conditional maximization either ecme algorithm to learn the model parameters our simulations demonstrate that when time delays are present td gpfa is able to correctly identify these delays and recover the latent space we then applied td gpfa to the activity of tens of neurons recorded simultaneously in the macaque motor cortex during a reaching task td gpfa is able to better describe the neural activity using a more parsimonious latent space than gpfa a method that has been used to interpret motor cortex data but does not account for time delays more broadly td gpfa can help to unravel the mechanisms underlying high dimensional time series data by taking into account physical delays in the system

eigenfaces vs fisherfaces recognition using class specific linear projection we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression taking a pattern classification approach we consider each pixel in an image as a coordinate in a high dimensional space we take advantage of the observation that the images of a particular face under varying illumination but fixed pose lie in a 3d linear subspace of the high dimensional image space if the face is a lambertian surface without shadowing however since faces are not truly lambertian surfaces and do indeed produce self shadowing images will deviate from this linear subspace rather than explicitly modeling this deviation we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation our projection method is based on fisher s linear discriminant and produces well separated classes in a low dimensional subspace even under severe variation in lighting and facial expressions the eigenface technique another method based on linearly projecting the image space to a low dimensional subspace has similar computational requirements yet extensive experimental results demonstrate that the proposed fisherface method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases

the classical shallow water equations symplectic geometry the classical shallow water equations express the change with time of the height h and the velocity of a 1 dimensional fluid t x h x 0 h x h x 0 they possess an infinite number of integrals of motion due to benney 1973 and can be written in hamiltonian form relative to a symplectic structure introduced by manin 1978 the present paper deals with their complete integrability up to the advent of shocks this is proved in the small under an extra assumption satisfied by most height velocity pairs that hh only at isolated points

learning visual spatial pooling by strong pca dimension reduction in visual modeling invariance properties of visual cells are often explained by a pooling mechanism in which outputs of neurons with similar selectivities to some stimulus parameters are integrated so as to gain some extent of invariance to other parameters for example the classical energy model of phase invariant v1 complex cells pools model simple cells preferring similar orientation but different phases prior studies such as independent subspace analysis have shown that phase invariance properties of v1 complex cells can be learned from spatial statistics of natural inputs however those previous approaches assumed a squaring nonlinearity on the neural outputs to capture energy correlation such nonlinearity is arguably unnatural from a neurobiological viewpoint but hard to change due to its tight integration into their formalisms moreover they used somewhat complicated objective functions requiring expensive computations for optimization in this study we show that visual spatial pooling can be learned in a much simpler way using strong dimension reduction based on principal component analysis this approach learns to ignore a large part of detailed spatial structure of the input and thereby estimates a linear pooling matrix using this framework we demonstrate that pooling of model v1 simple cells learned in this way even with nonlinearities other than squaring can reproduce standard tuning properties of v1 complex cells for further understanding we analyze several variants of the pooling model and argue that a reasonable pooling can generally be obtained from any kind of linear transformation that retains several of the first principal components and suppresses the remaining ones in particular we show how the classic wiener filtering theory leads to one such variant

robust and consistent estimates of autoregressive moving average parameters summary we aim at estimates of autoregressive moving average parameters which are simultaneously consistent at the nominal gaussian model and insensitive to outliers in the data the estimates may be viewed as a robust version of maximum likelihood estimates of order r and for purely autoregressive models reduce to a special case of generalized m estimates

feature selection for unsupervised learning in this paper we identify two issues involved in developing an automated feature subset selection algorithm for unlabeled data the need for finding the number of clusters in conjunction with feature selection and the need for normalizing the bias of feature selection criteria with respect to dimension we explore the feature selection problem and these issues through fssem feature subset selection using expectation maximization em clustering and through two different performance criteria for evaluating candidate feature subsets scatter separability and maximum likelihood we present proofs on the dimensionality biases of these feature criteria and present a cross projection normalization scheme that can be applied to any criterion to ameliorate these biases our experiments show the need for feature selection the need for addressing these two issues and the effectiveness of our proposed solutions

infinitesimal robustness for autoregressive processes we define the influence function and construct optimal robust estimators for autoregressive processes we show that the asymptotic bias caused by small contaminations of the marginals can be written as the integral of a certain function with respect to the contamination this function is called the influence function it is unique only up to an equivalence relation but there is a natural unique version which describes the limiting influence of an additional observation given the previous observations moreover with this version the asymptotic variance at the true model can be expressed in a simple form optimal robust estimators minimize this asymptotic variance under a constraint on the influence function as in the i i d case they are found by truncating a multiple of the influence function of the maximum likelihood estimator

compound rank k projections for bilinear analysis in many real world applications data are represented by matrices or high order tensors despite the promising performance the existing 2 d discriminant analysis algorithms employ a single projection model to exploit the discriminant information for projection making the model less flexible in this paper we propose a novel compound rank k projection crp algorithm for bilinear analysis the crp deals with matrices directly without transforming them into vectors and it therefore preserves the correlations within the matrix and decreases the computation complexity different from the existing 2 d discriminant analysis algorithms objective function values of crp increase monotonically in addition the crp utilizes multiple rank k projection models to enable a larger search space in which the optimal solution can be found in this way the discriminant ability is enhanced we have tested our approach on five data sets including uuim cvl pointing 04 usps and coil20 experimental results show that the performance of our proposed crp performs better than other algorithms in terms of classification accuracy

unsupervised 2d dimensionality reduction with adaptive structure learning in recent years unsupervised two dimensional 2d dimensionality reduction methods for unlabeled large scale data have made progress however performance of these degrades when the learning of similarity matrix is at the beginning of the dimensionality reduction process a similarity matrix is used to reveal the underlying geometry structure of data in unsupervised dimensionality reduction methods because of noise data it is difficult to learn the optimal similarity matrix in this letter we propose a new dimensionality reduction model for 2d image matrices unsupervised 2d dimensionality reduction with adaptive structure learning drasl instead of using a predetermined similarity matrix to characterize the underlying geometry structure of the original 2d image space our proposed approach involves the learning of a similarity matrix in the procedure of dimensionality reduction to realize a desirable neighbors assignment after dimensionality reduction we add a constraint to our model such that there are exact formula see text connected components in the final subspace to accomplish these goals we propose a unified objective function to integrate dimensionality reduction the learning of the similarity matrix and the adaptive learning of neighbors assignment into it an iterative optimization algorithm is proposed to solve the objective function we compare the proposed method with several 2d unsupervised dimensionality methods k means is used to evaluate the clustering performance we conduct extensive experiments on coil20 at t feret usps and yale data sets to verify the effectiveness of our proposed method

dimension selection for feature selection and dimension reduction with principal and independent component analysis this letter is concerned with the problem of selecting the best or most informative dimension for dimension reduction and feature extraction in high dimensional data the dimension of the data is reduced by principal component analysis subsequent application of independent component analysis to the principal component scores determines the most nongaussian directions in the lower dimensional space a criterion for choosing the optimal dimension based on bias adjusted skewness and kurtosis is proposed this new dimension selector is applied to real data sets and compared to existing methods simulation studies for a range of densities show that the proposed method performs well and is more appropriate for nongaussian data than existing methods

the hamiltonian structure of the maxwell vlasov equations morrison 25 has observed that the maxwell vlasov and poisson vlasov equations for a collisionless plasma can be written in hamiltonian form relative to a certain poisson bracket we derive another poisson structure for these equations by using general methods of symplectic geometry the main ingredients in our construction are the symplectic structure on the co adjoint orbits for the group of canonical transformations and the symplectic structure for the phase space of the electromagnetic field regarded as a gauge theory our poisson bracket satisfies the jacobi identity whereas morrison s does not 37 our construction also shows where canonical variables can be found and can be applied to the yang mills vlasov equations and to electromagnetic fluid dynamics

robust estimates for arma models abstract two new classes of robust estimates for arma models are introduced estimates based on residual autocovariances ra estimates and estimates based on truncated residual autocovariances tra estimates a heuristic derivation of the asymptotic normal distribution is given we also perform a monte carlo study to compare the robustness properties of these estimates with the least squares m and gm estimates in this study we consider observations that correspond to a gaussian model with additive outliers the monte carlo results show that ra and tra estimates compare favorably with respect to least squares m and gm estimates

orthogonal neighborhood preserving projections a projection based dimensionality reduction technique this paper considers the problem of dimensionality reduction by orthogonal projection techniques the main feature of the proposed techniques is that they attempt to preserve both the intrinsic neighborhood geometry of the data samples and the global geometry in particular we propose a method named orthogonal neighborhood preserving projections which works by first building an affinity graph for the data in a way that is similar to the method of locally linear embedding lle however in contrast with the standard lle where the mapping between the input and the reduced spaces is implicit onpp employs an explicit linear mapping between the two as a result handling new data samples becomes straightforward as this amounts to a simple linear transformation we show how we can define kernel variants of onpp as well as how we can apply the method in a supervised setting numerical experiments are reported to illustrate the performance of onpp and to compare it with a few competing methods

dimension reduction by local principal component analysis reducing or eliminating statistical redundancy between the components of high dimensional vector data enables a lower dimensional representation without significant loss of information recognizing the limitations of principal component analysis pca researchers in the statistics and neural network communities have developed nonlinear extensions of pca this article develops a local linear approach to dimension reduction that provides accurate representations and is fast to compute we exercise the algorithms on speech and image data and compare performance with pca and with neural network implementations of nonlinear pca we find that both nonlinear techniques can provide more accurate representations than pca and show that the local linear techniques outperform neural network implementations

face recognition using laplacianfaces we propose an appearance based face recognition method called the laplacianface approach by using locality preserving projections lpp the face images are mapped into a face subspace for analysis different from principal component analysis pca and linear discriminant analysis lda which effectively see only the euclidean structure of face space lpp finds an embedding that preserves local information and obtains a face subspace that best detects the essential face manifold structure the laplacianfaces are the optimal linear approximations to the eigenfunctions of the laplace beltrami operator on the face manifold in this way the unwanted variations resulting from changes in lighting facial expression and pose may be eliminated or reduced theoretical analysis shows that pca lda and lpp can be obtained from different graph models we compare the proposed laplacianface approach with eigenface and fisherface methods on three different face data sets experimental results suggest that the proposed laplacianface approach provides a better representation and achieves lower error rates in face recognition

general m estimates for contaminated p th order autoregressive processes consistency and asymptotic normality this work is concerned with simultaneous estimation of coefficients and a scale parameter of a p th order autoregressive process x t the observations are y t v t z t 1 v t x t where z t is a contaminating process and v t represents the proportion of contamination if x t or z t have heavy tails both least squares estimates and ordinary m estimates are seriously affected under general conditions we prove consistency and asymptotic normality of a general class of m estimates which contains some m estimates studied by denby and martin 6

laplacian eigenmaps for dimensionality reduction and data representation one of the central problems in machine learning and pattern recognition is to develop appropriate representations for complex data we consider the problem of constructing a representation for data lying on a low dimensional manifold embedded in a high dimensional space drawing on the correspondence between the graph laplacian the laplace beltrami operator on the manifold and the connections to the heat equation we propose a geometrically motivated algorithm for representing the high dimensional data the algorithm provides a computationally efficient approach to nonlinear dimensionality reduction that has locality preserving properties and a natural connection to clustering some potential applications and illustrative examples are discussed

rapid and brief communication two dimensional locality preserving projections 2dlpp with its application to palmprint recognition this paper proposes a novel algorithm for image feature extraction namely the two dimensional locality preserving projections 2dlpp which directly extracts the proper features from image matrices based on locality preserving criterion experimental results on the polyu palmprint database show the effectiveness of the proposed algorithm

discussion comment on two dimensional locality preserving projections 2dlpp with its application to palmprint recognition this paper proposes a novel algorithm for image feature extraction namely the two dimensional locality preserving projections 2dlpp which directly extracts the proper features from image matrices based on locality preserving criterion experimental results on the polyu palmprint database show the effectiveness of the proposed algorithm

the shallow water equations explicit solutions and superposition principle by using the hodograph method the author finds almost all the solutions of the classical shallow water equations the author also constructs an auto backlund transformation superposition principle on the set of these hodograph solutions and shows that this transformation is canonical relative to a symplectic form introduced by manin 1978

maximum likelihood estimation of order m for stationary stochastic processes sometimes the likelihood cannot be computed even introducing approximations whose effect is asymptotically negligible to overcome this problem for inference from a stochastic process we approximate the log likelihood by a sum whose generic term is the density function of the corresponding sample element conditional on the m most recent observations for some m s 0 general results on the asymptotic properties of the associated estimates are given and numerical comparisons with other estimators are carried out in special cases

conservation laws for shallow water waves on a sloping beach shallow water waves are governed by a pair of nonlinear partial differential equations we transfer the associated homogeneous and nonhomogeneous systems corresponding to constant and sloping depth respectively to the hodograph plane where we find all the nonsimple wave solutions and construct infinitely many polynomial conversation laws we also establish correspondence between conservation laws and hodograph solutions as well as backlund transformations by using the linear nature of the problems on the hodograph plane

a variance minimization criterion to feature selection using laplacian regularization in many information processing tasks one is often confronted with very high dimensional data feature selection techniques are designed to find the meaningful feature subset of the original features which can facilitate clustering classification and retrieval in this paper we consider the feature selection problem in unsupervised learning scenarios which is particularly difficult due to the absence of class labels that would guide the search for relevant information based on laplacian regularized least squares which finds a smooth function on the data manifold and minimizes the empirical loss we propose two novel feature selection algorithms which aim to minimize the expected prediction error of the regularized regression model specifically we select those features such that the size of the parameter covariance matrix of the regularized regression model is minimized motivated from experimental design we use trace and determinant operators to measure the size of the covariance matrix efficient computational schemes are also introduced to solve the corresponding optimization problems extensive experimental results over various real life data sets have demonstrated the superiority of the proposed algorithms