estimating the covariance matrix by signal subspace averaging an efficient algorithm is presented for estimating a covariance matrix consisting of a low rank signal term and a full rank noise term known apart from a scalar factor for each sample of the vector of sensor outputs the algorithm approximates in the least squares sense a rank one update of the covariance matrix under the side condition that the rank of the signal term remains bounded if the model noise is spatially colored the least squares approximation is preceded by spatial prewhitening it is shown that if the rank of the signal term is small compared to the number of sensors then the proposed algorithm requires substantially less computational work than conventional averaging some simulation results are included indicating that the proposed algorithm reduces the variance of some commonly used spectral estimators in off target directions without impairing their detection and resolution properties

the rise algorithm for recursive eigenspace decomposition a recursive method for generating the eigenvalues and eigenvectors of hermitian matrices is presented this algorithm is closely related to recursive iterative toeplitz eigenspace decomposition rite and thus may appropriately be named recursive iterative self adjoint eigenspace decomposition rise since it is formulated for the more general hermitian matrix rise may be performed at a cost of o n sup 3 multiplies per matrix order while rite requires o 4n sup 3 it is demonstrated that rise as is also true with rite is numerically stable and thus does not appreciably accumulate errors as the recursion progresses additionally a variation of rite is presented which allows the computation of the eigenvalues of successively smaller hermitian toeplitz matrices the initial presentation of rite only provided for the determination of the eigenvalues of successively larger matrices

spatial spectral filtering with linearly constrained minimum variance beamformers in this paper orthogonal low dimension models for broad band sources as observed by an array delay line structure are developed the models are termed broad band source representation spaces and are subspaces of the observation space they are generated from eigenstructures of source sample covariance matrices and represent sources efficiently in the 2nd order statistical sense for a single broad band source propagating in a pure delay environment and observed with an isotropic array an accurate indication of the model dimension is derived which is independent on the array configuration being a function of the observed source time bandwidth product only using these representations procedures are described for controlling the spatial and spectral response of linearly constrained minimum variance broad band array beamformers the procedures employ linear constraints to control complex gain characteristics over selected bands of frequency for multiple points of 3 d regions of source locations and do not require the use of steering delays the constraints are derived from source representation spaces and are termed eigenvector constraints simulations are presented which illustrate the effectiveness of the new linear constraints

detection of signals by information theoretic criteria a new approach is presented to the problem of detecting the number of signals in a multichannel time series based on the application of the information theoretic criteria for model selection introduced by akaike aic and by schwartz and rissanen mdl unlike the conventional hypothesis testing based approach the new approach does not requite any subjective threshold settings the number of signals is obtained merely by minimizing the aic or the mdl criteria simulation results that illustrate the performance of the new method for the detection of the number of signals received by a sensor array are presented

probabilistic visual learning for object representation we present an unsupervised technique for visual learning which is based on density estimation in high dimensional spaces using an eigenspace decomposition two types of density estimates are derived for modeling the training data a multivariate gaussian for unimodal distributions and a mixture of gaussians model for multimodal distributions those probability densities are then used to formulate a maximum likelihood estimation framework for visual search and target detection for automatic object recognition and coding our learning technique is applied to the probabilistic visual modeling detection recognition and coding of human faces and nonrigid objects such as hands

scaling for orthogonality in updating algorithms where orthogonal transformations are accumulated it is important to preserve the orthogonality of the product in the presence of rounding error moonen et al ibid vol 39 p 1911 13 1991 have pointed out that simply normalizing the columns of the product tends to preserve orthogonality though not as degroat ibid vol 39 p 1913 14 1991 points out to working precision the authors discuss the previous work and give an analysis of the phenomenon

filtering and frequency estimation using perturbation formulas a new method for updating the svd is introduced based on perturbation formulas the complexity of the method is o n sup 2 applications are made to frequency estimation and filtering

urv esprit for tracking time varying signals esprit is an algorithm for determining the fixed directions of arrival of a set of narrowband signals at an array of sensors unfortunately its computational burden makes it unsuitable for real time processing of signals with time varying directions of arrival the authors develop a new implementation of esprit that has potential for real time processing it is based on a rank revealing urv decomposition rather than the eigendecomposition or singular value decomposition used in previous esprit algorithms the authors demonstrate its performance on simulated data representing both constant and time varying signals they find that the urv based esprit algorithm is effective for estimating time varying directions of arrival at considerable computational savings over the svd based algorithm

incremental learning of discriminant common vectors for feature extraction abstract discriminant common vectors dcv which can effectively extract the features of face images is a recently proposed algorithm to overcome the small sample size sss problem encountered by linear discriminant analysis lda its numerical accuracy is high and computational complexity is low however the dcv algorithm is not suitable for online training problems in order to address this problem an incremental dcv idcv method is developed in this paper the idcv algorithm can incrementally learn the optimal projection matrix instead of recomputing the dcv again when new sample is added into the training set theoretical analysis denotes that idcv is much more efficient that dcv experiments on orl pie and ar face databases demonstrate the efficiency of our proposed idcv algorithm over the original batch dcv algorithm

orthogonal eigensubspace estimation using neural networks we present a neural network nn approach for simultaneously estimating all or some of the orthogonal eigenvectors of a symmetric nonindefinite matrix corresponding to its repeated minimum in magnitude eigenvalue this problem has its origin in the constrained minimization framework and has extensive applications in signal processing we recast this problem into the nn framework by constructing an appropriate energy function which the nn minimizes the nn is of feedback type with the neurons having sigmoidal activation function the proposed approach is analyzed to characterize the nature of the minimizers of the energy function the main result is that the matrix w is a minimizer of the energy function if and only if the columns of w are the orthogonal eigenvectors with a given norm corresponding to the smallest eigenvalue of the given matrix further all minimizers are global minimizers bounds on the integration time step that is required to numerically solve the system of differential equations which define the dynamics of the nn have also been derived results of computer simulations are presented to support our analysis

adaptive estimation of eigensubspace in a recent work we recast the problem of estimating the minimum eigenvector eigenvector corresponding to the minimum eigenvalue of a symmetric positive definite matrix into a neural network framework we now extend this work using an inflation technique to estimate all or some of the orthogonal eigenvectors of the given matrix based on these results we form a cost function for the finite data case and derive a newton based adaptive algorithm the inflation technique leads to a highly modular and parallel structure for implementation the computational requirement of the algorithm is o n sup 2 n being the size of the covariance matrix we also present a rigorous convergence analysis of this adaptive algorithm the algorithm is locally convergent and the undesired stationary points are unstable computer simulation results are provided to compare its performance with that of two adaptive subspace estimation methods proposed by yang and kaveh 1988 and an improved version of one of them for stationary and nonstationary signal scenarios the results show that the proposed approach performs identically to one of them and is significantly superior to the remaining two

the signal subspace approach for multiple wide band emitter location the rational vector space generalization of the signal subspace approach is presented and applied to the estimation of multiple wide band emitter locations from the signals received at multiple sensors the signal subspace and array manifold concepts first introduced by schmidt are generalized to rational vector space these concepts are used to develop the rational signal subspace theory and prove the signal subspace theorem on which signal subspace algorithms are based the theory is applied in discrete time to derive a class of rational signal subspace algorithms for source location and spectral estimation using unit circle eigendecomposition of multivariate rational models of sensor outputs simulation results are presented for an algorithm in this class including sample statistics from monte carlo trials and comparisons with the cramer rao bound

clustered blockwise pca for representing visual data principal component analysis pca is extensively used in computer vision and image processing since it provides the optimal linear subspace in a least square sense it has been used for dimensionality reduction and subspace analysis in various domains however its scalability is very limited because of its inherent computational complexity we introduce a new framework for applying pca to visual data which takes advantage of the spatio temporal correlation and localized frequency variations that are typically found in such data instead of applying pca to the whole volume of data complete set of images we partition the volume into a set of blocks and apply pca to each block then we group the subspaces corresponding to the blocks and merge them together as a result we not only achieve greater efficiency in the resulting representation of the visual data but also successfully scale pca to handle large data sets we present a thorough analysis of the computational complexity and storage benefits of our approach we apply our algorithm to several types of videos we show that in addition to its storage and speed benefits the algorithm results in a useful representation of the visual data

achieving robust face recognition from video by combining a weak photometric model and a learnt generic face invariant in spite of over two decades of intense research illumination and pose invariance remain prohibitively challenging aspects of face recognition for most practical applications the objective of this work is to recognize faces using video sequences both for training and recognition input in a realistic unconstrained setup in which lighting pose and user motion pattern have a wide variability and face images are of low resolution the central contribution is an illumination invariant which we show to be suitable for recognition from video of loosely constrained head motion in particular there are three contributions i we show how a photometric model of image formation can be combined with a statistical model of generic face appearance variation to exploit the proposed invariant and generalize in the presence of extreme illumination changes ii we introduce a video sequence re illumination algorithm to achieve fine alignment of two video sequences and iii we use the smoothness of geodesically local appearance manifold structure and a robust same identity likelihood to achieve robustness to unseen head poses we describe a fully automatic recognition system based on the proposed method and an extensive evaluation on 323 individuals and 1474 video sequences with extreme illumination pose and head motion variation our system consistently achieved a nearly perfect recognition rate over 99 7 on all four databases

multiple emitter location and signal parameter estimation processing the signals received on an array of sensors for the location of the emitter is of great enough interest to have been treated under many special case assumptions the general problem considers sensors with arbitrary locations and arbitrary directional characteristics gain phase polarization in a noise interference environment of arbitrary covariance matrix this report is concerned first with the multiple emitter aspect of this problem and second with the generality of solution a description is given of the multiple signal classification music algorithm which provides asymptotically unbiased estimates of 1 number of incident wavefronts present 2 directions of arrival doa or emitter locations 3 strengths and cross correlations among the incident waveforms 4 noise interference strength examples and comparisons with methods based on maximum likelihood ml and maximum entropy me as well as conventional beamforming are included an example of its use as a multiple frequency estimator operating on time series is included

conjugate gradient eigenstructure tracking for adaptive spectral estimation a conjugate gradient iteration is derived that converges to the set of r dominant subdominant eigenpairs this iteration is used to construct two eigenstructure tracking algorithms that track the r dimensional dominant or subdominant subspaces of time varying data or data covariance matrices the two eigenstructure tracking algorithms have update complexities o m sup 2 r and the other o mr sup 2 where m is the data dimension the algorithms are customized to solve high resolution temporal and spatial frequency tracking problems they are compared with existing techniques by tying into published simulation based performance tests the algorithms demonstrate rapid convergence and tracking characteristics at a competitive cost

incremental learning of bidirectional principal components for face recognition recently bidirectional principal component analysis bdpca has been proven to be an efficient tool for pattern recognition and image analysis encouraging experimental results have been reported and discussed in the literature however bdpca has to be performed in batch mode it means that all the training data has to be ready before we calculate the projection matrices if there are additional samples need to be incorporated into an existing system it has to be retrained with the whole updated training set moreover the scatter matrices of bdpca are formulated as the sum of k samples size image covariance matrices this leads to the incremental learning directly on the scatters impossible thus it presents new challenge for on line training in fact there are two major reasons for building incremental algorithms the first reason is that in some cases when the number of training images is very large the batch algorithm cannot process the entire training set due to large computational or space requirements of the batch approach the second reason is when the learning algorithm is supposed to operate in a dynamical settings that all the training data is not given in advance and new training samples may arrive at any time and they have to be processed in an on line manner through matricizations of third order tensor we successfully transfer the eigenvalue decomposition problem of scatters to the singular value decomposition svd of corresponding unfolded matrices followed by complexity and memory analysis on the novel algorithm a theoretical clue for selecting suitable dimensionality parameters without losing classification information is also presented in this paper experimental results on feret and cmu pie pose illumination and expression databases show that the ibdpca algorithm gives a close approximation to the bdpca method but using less time

perturbation scheme for online learning of features incremental principal component analysis a new incremental online feature extraction approach is proposed based on principal component analysis in conjunction with perturbation theory which for its validity requires the perturbation parameter to be small our approach is found to be computationally more efficient in comparison to batch method which for its applicability requires simultaneous availability of all observations for computation of features it is found on the basis of numerical experiments that the results based on our approach besides being in good agreement with the batch method and other incremental methods are also computationally more efficient to demonstrate the efficacy of the proposed scheme experiments have been performed on randomly generated datasets as well as on low and high dimensional datasets i e uci and face datasets which are available in public domain

estimation of phase angles from the cross spectral matrix the so called signal subspace methods in spatial signal processing have been used up to now in the case of steering vectors depending upon very few parameters for instance they are used currently for steering vectors of the form 1 ei e2i ei n 1 twhich occur in the case of plane waves and a linear array of n equispaced sensors we show by providing a suitable algorithm that these methods can be extended to the general case of steering vectors of the form 1 ei 2 ei 3 ei n twhere the phase angles 2 n are treated as free parameters this kind of situation occurs for instance in the case of randomly distorted wavefronts and or flexible antennas of unknown geometry when the fluctuations of the wavefronts and of the antenna are very slow with respect to the time constants used for the estimation of the cross spectral matrices as usual in signal subspace methods the proposed algorithm works very well when the problem presents a strong degree of overdetermination many more sensors than sources and the cross spectral matrix is estimated with good precision

plane rotation based evd updating schemes for efficient subspace tracking we present new algorithms based on plane rotations for tracking the eigenvalue decomposition evd of a time varying data covariance matrix these algorithms directly produce eigenvectors in orthonormal form and are well suited for the application of subspace methods to nonstationary data after recasting evd tracking as a simplified rank one evd update problem computationally efficient solutions are obtained in two steps first a new kind of parametric perturbation approach is used to express the eigenvector update as an unimodular orthogonal transform which is represented in exponential matrix form in terms of a reduced set of small unconstrained parameters second two approximate decompositions of this exponential matrix into products of plane or givens rotations are derived one of which being previously unknown these decompositions lead to new plane rotation based evd updating schemes proteus whose main feature is the use of plane rotations for updating the eigenvectors thereby preserving orthonormality finally the proteus schemes are used to derive new evd trackers whose convergence and numerical stability are investigated via simulations one algorithm can track all the signal subspace evd components in only o lm operations where l and m respectively denote the data vector and signal subspace dimensions while achieving a performance comparable to an exact evd approach and maintaining perfect orthonormality of the eigenvectors the new algorithms show no signs of error buildup

weighted resolvent estimates for volterra operators on unbounded intervals the interval may be bounded or unbounded including the case of the whole real line and the kernel v r q is supposed to be such that v maps x into x we shall make precise assumptions later the case of a bounded interval f with x w y and v q is classical in this case the operator v is bounded and its spectrum consists of 0 in other words the spectral radius of v is zero a standard proof of these two facts is to prove that for 13 0 the neumann series for the inverse of azv converges with respect to the operator norm the purpose of this note is to study the case of a possibly unbounded

incremental discriminant analysis of canonical correlations for action recognition human action recognition from video sequences is a challenging problem due to the large changes of human appearance in the cases of partial occlusions non rigid deformations and high irregularities it is difficult to collect a large set of training samples to learn the discriminative model with covering all possible variations of an action in this paper we propose an online recognition method namely incremental discriminant analysis of canonical correlations idcc in which the discriminative model is incrementally updated to capture the changes of human appearance and thereby facilitates the recognition task in changing environments as the training sets are acquired sequentially instead of being given completely in advance our method is able to compute a new discriminant matrix by updating the existing one using the eigenspace merging algorithm furthermore we integrate our method into the graph based semi supervised learning method linear neighbor propagation to deal with the limited labeled training data experimental results on both weizmann and kth action data sets show that our method performs better than state of the art methods on accuracy and efficiency

a framework for weighted fusion of multiple statistical models of shape and appearance this paper presents a framework for weighted fusion of several active shape and active appearance models the approach is based on the eigenspace fusion method proposed by hall et al which has been extended to fuse more than two weighted eigenspaces using unbiased mean and covariance matrix estimates to evaluate the performance of fusion a comparative assessment on segmentation precision as well as facial verification tests are performed using the ar equinox and xm2vts databases based on the results it is concluded that the fusion is useful when the model needs to be updated online or when the original observations are absent

estimating the parameters of exponentially damped sinusoids and pole zero modeling in noise we have presented techniques 1 6 based on linear prediction lp and singular value decomposition svd for accurate estimation of closely spaced frequencies of sinusoidal signals in noise in this note we extend these techniques to estimate the parameters of exponentially damped sinusoidal signals in noise the estimation procedure presented here makes use of backward prediction in addition to svd first the method is applied to data consisting of one and two exponentially damped sinusoids the choice of one and two signal components facilitates the comparison of estimation error in pole damping factors and pole frequencies to the appropriate cramer rao cr bounds and to traditional methods of linear prediction second our method is applied to an example due to steiglitz 8 in which the data consists of noisy values of the impulse response samples composed of many exponentially damped sinusoids of a linear system having both poles and zeros the poles of the system are accurately determined by our method and the zeros are obtained subsequently using shanks method

esprit a subspace rotation approach to estimation of parameters of cisoids in noise the application of a subspace invariance approach esprit to the estimation of parameters frequencies and powers of cisoids in noise is described esprit exploits an underlying rotational invariance of signal subspaces spanned by two temporally displaced data sets the new approach has several advantages including improved resolution over pisarenko s technique for harmonic retrieval

on resolving 2m 1 narrow band signals with an m sensor uniform linear array the problem of determining the maximum number of narrowband signals whose parameters can be estimated with a linear array of m equally spaced sensors is examined while this number previously has been taken to be m 1 when the signals are mutually uncorrelated it is shown how to estimate directions and amplitudes for as many as 2m 1 signals this over twofold increase is accomplished by retaining snapshot to snapshot phase information usually lost in algorithms based on spatial correlation matrices the approach uses length 2m real signal vectors rather than the usual m complex vectors it is shown that 2m of these real vectors are linearly independent with probability one and thus in the presence of additive white noise the parameters of 2m 1 signals can be estimated an algorithm for determining directions and amplitudes is presented because of the algorithm s computational complexity its application is limited to small m and low time bandwidth products

efficient numerically stabilized rank one eigenstructure updating signal processing rank one eigenvalue decomposition evd updating is well suited to the problem of tracking time varying subspaces previously published rank one evd updating algorithms suffer from a linear buildup of roundoff error which makes them impractical for a large number of recursive updates in the present work the authors develop a numerical stabilization technique which eliminates the error buildup problem in a computationally efficient manner and makes the rank one evd update a practical numerical tool for online computation a simplified eigenvalue iteration is also given it reduces the complexity of the algorithm somewhat as well as the computation time simulations are presented to illustrate numerical performance

incremental generalized discriminative common vectors for image classification subspace based methods have become popular due to their ability to appropriately represent complex data in such a way that both dimensionality is reduced and discriminativeness is enhanced several recent works have concentrated on the discriminative common vector dcv method and other closely related algorithms also based on the concept of null space in this paper we present a generalized incremental formulation of the dcv methods which allows the update of a given model by considering the addition of new examples even from unseen classes having efficient incremental formulations of well behaved batch algorithms allows us to conveniently adapt previously trained classifiers without the need of recomputing them from scratch the proposed generalized incremental method has been empirically validated in different case studies from different application domains faces objects and handwritten digits considering several different scenarios in which new data are continuously added at different rates starting from an initial model

new parametric measures of information in this paper methods are presented for obtaining parametric measures of information from the non parametric ones and from information matrices the properties of these measures are examined the one dimensional parametric measures which are derived from the non parametric are superior to fisher s information measure because they are free from regularity conditions but if we impose the regularity conditions of the fisherian theory of information these measures become linear functions of fisher s measure

implementation of adaptive array algorithms some new efficient and numerically stable algorithms for the recursive solution of matrix problems arising in optimal beam forming and direction finding are described and analyzed the matrix problems considered are systems of linear equations and spectral decomposition while recursive solution procedures based on the matrix inversion lemma may be unstable ours are stable furthermore these algorithms are extremely fast

an information fusion framework for robust shape tracking existing methods for incorporating subspace model constraints in shape tracking use only partial information from the measurements and model distribution we propose a unified framework for robust shape tracking optimally fusing heteroscedastic uncertainties or noise from measurement system dynamics and a subspace model the resulting nonorthogonal subspace projection and fusion are natural extensions of the traditional model constraint using orthogonal projection we present two motion measurement algorithms and introduce alternative solutions for measurement uncertainty estimation we build shape models offline from training data and exploit information from the ground truth initialization online through a strong model adaptation our framework is applied for tracking in echocardiograms where the motion estimation errors are heteroscedastic in nature each heart has a distinct shape and the relative motions of epicardial and endocardial borders reveal crucial diagnostic features the proposed method significantly outperforms the existing shape space constrained tracking algorithm due to the complete treatment of heteroscedastic uncertainties the strong model adaptation and the coupled tracking of double contours robust performance is observed even on the most challenging cases

adaptation dynamics of the spherical subspace tracker l ljung s 1977 method for analyzing recursive stochastic algorithms is used to formulate a projection operator ordinary differential equation ode the ode describes the expected convergence dynamics of a noniterative spherical subspace tracker the subspace ode is a riccati equation defined over the manifold of rank r projection matrices in c sup nxn a lyapunov function is defined that is shown to have global maximum and minimum at the signal and noise subspaces respectively by taking a derivative of the lyapunov function along any trajectory it is shown that the dynamics force all trajectories to converge to the signal subspace if the sign of the derivative is changed all trajectories will converge to the noise subspace

merging and splitting eigenspace models we present new deterministic methods that given two eigenspace models each representing a set of n dimensional observations will 1 merge the models to yield a representation of the union of the sets and 2 split one model from another to represent the difference between the sets as this is done we accurately keep track of the mean here we give a theoretical derivation of the methods empirical results relating to the efficiency and accuracy of the techniques and three general applications including the construction of gaussian mixture models that are dynamically updateable

noniterative subspace tracking a rank one spherical subspace update that is appropriate for subspace based methods like music and minimum norm is introduced this noniterative highly parallel numerically stabilized subspace update is closely related to rank one eigenstructure updating however a rank one subspace update involves less computation than simple rank one correlation accumulation moreover the frequency tracking capabilities of the noniterative subspace update are virtually identical to and in some case more robust than the more computationally expensive eigen based methods

singular value decomposition and improved frequency estimation using linear prediction linear prediction based lp methods for fitting multiple sinusoid signal models to observed data such as the forward backward fblp method of nuttall 5 and ulrych and clayton 6 are very ill conditioned the locations of estimated spectral peaks can be greatly affected by a small amount of noise because of the appearance of outliers lp estimation of frequencies can be greatly improved at low snr by singular value decomposition svd of the lp data matrix the improved performance at low snr is also better than that obtained by using the eigenvector corresponding to the minimum eigenvalue of the correlation matrix as is done in pisarenko s method and its variants

boosted manifold principal angles for image set based recognition in this paper we address the problem of classifying vector sets we motivate and introduce a novel method based on comparisons between corresponding vector subspaces in particular there are two main areas of novelty i we extend the concept of principal angles between linear subspaces to manifolds with arbitrary nonlinearities ii it is demonstrated how boosting can be used for application optimal principal angle fusion the strengths of the proposed method are empirically demonstrated on the task of automatic face recognition afr in which it is shown to outperform state of the art methods in the literature

a reverse formulation of the rise algorithm the recently developed recursive iterative self adjoint eigenspace rise decomposition algorithm recursively computes the complete eigenspace decomposition of successively larger hermitian matrices however some practical applications also require that the decomposition of successively smaller matrices be computed a modification to the rise algorithm is presented that makes it possible to run this algorithm backward on successively smaller hermitian matrices this important modification increases the number of practical applications of this algorithm

a quasi newton adaptive algorithm for generalized symmetric eigenvalue problem we first recast the generalized symmetric eigenvalue problem where the underlying matrix pencil consists of symmetric positive definite matrices into an unconstrained minimization problem by constructing an appropriate cost function we then extend it to the case of multiple eigen vectors using an inflation technique based on this asymptotic formulation we derive a quasi newton based adaptive algorithm for estimating the required generalized eigen vectors in the data case the resulting algorithm is modular and parallel and it is globally convergent with probability one we also analyze the effect of inexact inflation on the convergence of this algorithm and that of inexact knowledge of one of the matrices in the pencil on the resulting eigenstructure simulation results demonstrate that the performance of this algorithm is almost identical to that of the rank one updating algorithm of karasalo 1986 further the performance of the proposed algorithm has been found to remain stable even over 1 million updates without suffering from any error accumulation problems

a note on efficient numerically stabilized rank one eigenstructure updating and reply in the above named paper r d degroat and r a roberts see ibid vol 38 no 2 p 301 16 feb 1990 developed a reorthogonalization scheme for stabilizing eigenstructure updating algorithms the commenters show that only part of this scheme namely the renormalization is essential for the stability so that a cheaper scheme with roughly half as many computations can perform equally well degroat addresses two points raised by the commenters work

incremental kernel principal component analysis the kernel principal component analysis kpca has been applied in numerous image related machine learning applications and it has exhibited superior performance over previous approaches such as pca however the standard implementation of kpca scales badly with the problem size making computations for large problems infeasible also the batch nature of the standard kpca computation method does not allow for applications that require online processing this has somewhat restricted the domains in which kpca can potentially be applied this paper introduces an incremental computation algorithm for kpca to address these two problems the basis of the proposed solution lies in computing incremental linear pca in the kernel induced feature space and constructing reduced set expansions to maintain constant update speed and memory usage we also provide experimental results which demonstrate the effectiveness of the approach

on minimum fisher information distributions with restricted support and fixed variance fisher information is of key importance in estimation theory it also serves in inference problems as well as in the interpretation of many physical processes the mean squared estimation error for the location parameter of a distribution is bounded by the inverse of the fisher information associated with this distribution in this paper we look for minimum fisher information distributions with a restricted support more precisely we study the problem of minimizing the fisher information in the set of distributions with fixed variance defined on a bounded subset s of r or on the positive real line we show that the solutions of the underlying differential equation can be expressed in terms of whittaker functions then in the two considered cases we derive the explicit expressions of the solutions and investigate their behavior we also characterize the behavior of the minimum fisher information as a function of the imposed variance

on the performance of signal subspace processing part i narrow band systems this paper presents an analytical evaluation of detection determination of the number of sources and estimation performances of narrow band signal subspace processing for multiple source direction finding the probabilities of underestimating and overestimating the number of sources are derived under asymptotic conditions and around the threshold regions in terms of the choice of a penalty function and signal noise and array parameters for the cases of at most two closely spaced sources in the spatially white noise a scalar measure is introduced for the evaluation of the quality of the estimated signal subspace based on the statistics of this measure performance thresholds are demonstrated for the signal to noise ratio angle separation and correlation between two equipowered sources

eigenspace updating for non stationary process and its application to face recognition in this paper we introduce a novel approach to modeling non stationary random processes given a set of training samples sequentially we can iteratively update the eigenspace to manifest the current statistics provided by each new sample the updated eigenspace is derived based more on recent samples and less on older samples controlled by a number of decay parameters extensive study has been performed on how to choose these decay parameters other existing eigenspace updating algorithms can be regarded as special cases of our algorithm we show the effectiveness of the proposed algorithm with both synthetic data and practical applications on face recognition significant improvements have been observed on face images with different variations such as pose expression and illumination variations we expect the proposed algorithm to have other applications in active recognition and modeling as well

connections of generalized divergence measures with fisher information matrix abstract in this paper parametric measures of information for the multiparameter case are obtained using csiszar s nonparametric measure of information as well as continuous and differentiable functions thereof the same idea is applied to some of the unified r s divergence measures presented in taneja relationships of the new parametric measures with fisher s information matrix are derived and their connections with the cramer rao inequality is established this work generalizes the results of kagan aggarwal and ferentinos and papioannou

online kernel principal component analysis a reduced order model kernel principal component analysis kernel pca is an elegant nonlinear extension of one of the most used data analysis and dimensionality reduction techniques the principal component analysis in this paper we propose an online algorithm for kernel pca to this end we examine a kernel based version of oja s rule initially put forward to extract a linear principal axe as with most kernel based machines the model order equals the number of available observations to provide an online scheme we propose to control the model order we discuss theoretical results such as an upper bound on the error of approximating the principal functions with the reduced order model we derive a recursive algorithm to discover the first principal axis and extend it to multiple axes experimental results demonstrate the effectiveness of the proposed approach both on synthetic data set and on images of handwritten digits with comparison to classical kernel pca and iterative kernel pca

unique solvability of nonlinear volterra equations in weighted spaces abstract we investigate integral equations of the form x t g t t f t s x s ds in general this equation is history dependent so one needs to give an initial condition on 0 in order to obtain a unique solution by introducing a weight function on r we can single out a class of admissible solutions and give conditions for the unique solvability of in this restricted class we also study some fredholm equations on these weighted spaces in addition we also treat a class of equations of the first kind for which similar conclusions can be drawn

modal decomposition signal subspace algorithms a new method of estimating the locations of multiple wide band emitters arriving at an array of sensors is presented in this paper the wide band emitter location problem is treated as a separable multidimensional spectrum estimation problem by modal decomposition of the estimated spectral factor or spectral density matrix at the poles of the emitters this modal decomposition method extends conventional sinusoidal separation approaches to wide band emitters several methods of modal decomposition are presented and the signal subspace approach of schmidt 2 and su and morf 3 4 is extended and applied to the decomposed spectral residues for determination of emitter locations the method is asymptotically exact and provides the capability to resolve correlated or even coherent emitters

canonical correlation analysis of video volume tensors for action categorization and detection this paper addresses a spatiotemporal pattern recognition problem the main purpose of this study is to find a right representation and matching of action video volumes for categorization a novel method is proposed to measure video to video volume similarity by extending canonical correlation analysis cca a principled tool to inspect linear relations between two sets of vectors to that of two multiway data arrays or tensors the proposed method analyzes video volumes as inputs avoiding the difficult problem of explicit motion estimation required in traditional methods and provides a way of spatiotemporal pattern matching that is robust to intraclass variations of actions the proposed matching is demonstrated for action classification by a simple nearest neighbor classifier we moreover propose an automatic action detection method which performs 3d window search over an input video with action exemplars the search is speeded up by dynamic learning of subspaces in the proposed cca experiments on a public action data set kth and a self recorded hand gesture data showed that the proposed method is significantly better than various state of the art methods with respect to accuracy our method has low time complexity and does not require any major tuning parameters

adaptive eigendecomposition of data covariance matrices based on first order perturbations in this paper new algorithms for adaptive eigendecomposition of time varying data covariance matrices are presented the algorithms are based on a first order perturbation analysis of the rank one update for covariance matrix estimates with exponential windows different assumptions on the eigenvalue structure lead to three distinct algorithms with varying degrees of complexity a stabilization technique is presented and both issues of initialization and computational complexity are discussed computer simulations indicate that the new algorithms can achieve the same performance as a direct approach in which the exact eigendecomposition of the updated sample covariance matrix is obtained at each iteration previous algorithms with similar performance require o lm sup 2 complex operations per iteration where l and m respectively denote the data vector and signal subspace dimensions and involve either some form of gram schmidt orthogonalization or a nonlinear eigenvalue search the new algorithms have parallel structures sequential operation counts of order o lm or less and do not involve any of the above steps one particular algorithm can be used to update the complete signal subspace eigenstructure in 5lm complex operations this represents an order of magnitude improvement in computational complexity over existing algorithms with similar performance finally a simplified local convergence analysis of one of the algorithms shows that it is stable and converges in the mean to the true eigendecomposition the convergence is geometrical and is characterized by a single time constant

efficient calculation of primary images from a set of images a set of images is modeled as a stochastic process and karhunen loeve expansion is applied to extract the feature images although the size of the correlation matrix for such a stochastic process is very large we show the way to calculate the eigenvectors when the rank of the correlation matrix is not large we also propose an iterative algorithm to calculate the eigenvectors which save computation time andc omputer storage requirements this iterative algorithm gains its efficiency from the fact that only a significant set of eigenvectors are retained at any stage of iteration simulation results are also presented to verify these methods

modeling the manifolds of images of handwritten digits this paper describes two new methods for modeling the manifolds of digitized images of handwritten digits the models allow a priori information about the structure of the manifolds to be combined with empirical data accurate modeling of the manifolds allows digits to be discriminated using the relative probability densities under the alternative models one of the methods is grounded in principal components analysis the other in factor analysis both methods are based on locally linear low dimensional approximations to the underlying data manifold links with other methods that model the manifold are discussed

hallucinating optimal high dimensional subspaces abstract linear subspace representations of appearance variation are pervasive in computer vision this paper addresses the problem of robustly matching such subspaces computing the similarity between them when they are used to describe the scope of variations within sets of images of different possibly greatly so scales a naive solution of projecting the low scale subspace into the high scale image space is described first and subsequently shown to be inadequate especially at large scale discrepancies a successful approach is proposed instead it consists of i an interpolated projection of the low scale subspace into the high scale space which is followed by ii a rotation of this initial estimate within the bounds of the imposed downsampling constraint the optimal rotation is found in the closed form which best aligns the high scale reconstruction of the low scale subspace with the reference it is compared to the method is evaluated on the problem of matching sets of i face appearances under varying illumination and ii object appearances under varying viewpoint using two large data sets in comparison to the naive matching the proposed algorithm is shown to greatly increase the separation of between class and within class similarities as well as produce far more meaningful modes of common appearance on which the match score is based

information and random censoring fisher and divergence type measures of information in the area of random censoring are introduced and compared with the measures of hollander proschan and sconing the basic properties of statistical information theory are established for these measures of information the winners are the classical measures of information