binet cauchy kernels on dynamical systems and its application to the analysis of dynamic scenes we propose a family of kernels based on the binet cauchy theorem and its extension to fredholm operators our derivation provides a unifying framework for all kernels on dynamical systems currently used in machine learning including kernels derived from the behavioral framework diffusion processes marginalized kernels kernels on graphs and the kernels on sets arising from the subspace angle approach in the case of linear time invariant systems we derive explicit formulae for computing the proposed binet cauchy kernels by solving sylvester equations and relate the proposed kernels to existing kernels based on cepstrum coefficients and subspace angles we show efficient methods for computing our kernels which make them viable for the practitioner besides their theoretical appeal these kernels can be used efficiently in the comparison of video sequences of dynamic scenes that can be modeled as the output of a linear time invariant dynamical system one advantage of our kernels is that they take the initial conditions of the dynamical systems into account as a first example we use our kernels to compare video sequences of dynamic textures as a second example we apply our kernels to the problem of clustering short clips of a movie experimental evidence shows superior performance of our kernels

probability product kernels the advantages of discriminative learning algorithms and kernel machines are combined with generative modeling using a novel kernel between distributions in the probability product kernel data points in the input space are mapped to distributions over the sample space and a general inner product is then evaluated as the integral of the product of pairs of distributions the kernel is straightforward to evaluate for all exponential family models such as multinomials and gaussians and yields interesting nonlinear kernels furthermore the kernel is computable in closed form for latent distributions such as mixture models hidden markov models and linear dynamical systems for intractable models such as switching linear dynamical systems structured mean field approximations can be brought to bear on the kernel evaluation for general distributions even if an analytic expression for the kernel is not feasible we show a straightforward sampling method to evaluate it thus the kernel permits discriminative learning methods including support vector machines to exploit the properties metrics and invariances of the generative models we infer from each datum experiments are shown using multinomial models for text hidden markov models for biological data sets and linear dynamical systems for time series data

dynamic stability of a rectangular plate subjected to distributed in plane dynamic force dynamic instability of a rectangular plate subjected to an in plane sinusoidally varying force which varies along the edges is analyzed the thin plate small deflection theory is used the boundary conditions of the plate consist of combinations of simply supported edge and clamped edge unstable regions are presented for various boundary conditions of the plate and loading conditions

learning with jensen tsallis kernels jensen type jensen shannon js and jensen tsallis kernels were first proposed by martins et al 2009 these kernels are based on js divergences that originated in the information theory in this paper we extend the jensen type kernels on probability measures to define positive definite kernels on euclidean space we show that the special cases of these kernels include dot product kernels since jensen type divergences are multidistribution divergences we propose their multipoint variants and study spectral clustering and kernel methods based on these we also provide experimental studies on benchmark image database and gene expression database that show the benefits of the proposed kernels compared with the existing kernels the experiments on clustering also demonstrate the use of constructing multipoint similarities

instability of parametric dynamic systems with non uniform damping abstract a unified way of looking at parametric and combination resonances in systems with periodic coefficients and different amounts of damping in the various modes of vibration is presented the stability boundaries of the coupled mathieu equations are determined by the harmonic balance method fourier series with periods 2t and t being assumed the basic characteristics of the solution are discussed and the method is applied to multiple degree of freedom dynamic systems the destabilizing effect on the combination resonances is obtained by the present method

approximation techniques for the transformation of fuzzy sets into random sets with the recent rising of numerous theories for dealing with uncertain pieces of information the problem of connection between different frames has become an issue in particular questions such as how to combine fuzzy sets with belief functions or probability measures often emerge the alternative is either to define transformations between theories or to use a general or unified framework in which all these theories can be framed random set theory has been proposed as such a unified framework in which at least probability theory evidence theory possibility theory and fuzzy set theory can be represented whereas the transformations of belief functions or probability distributions into random sets are trivial the transformations of fuzzy sets or possibility distributions into random sets lead to some issues this paper is concerned with the transformation of fuzzy membership functions into random sets in practice this transformation involves the creation of a large number of focal elements subsets with non null probability based on the a cuts of the fuzzy membership functions in order to keep a computationally tractable fusion process the large number of focal elements needs to be reduced by approximation techniques in this paper we propose three approximation techniques and compare them to classical approximations techniques used in evidence theory the quality of the approximations is quantified using a distance between two random sets

dimensionality reduction for data of unknown cluster structure dimensionality reduction that preserves certain characteristics of data is needed for numerous reasons in this work we focus on data coming from a mixture of gaussian distributions and we propose a method that preserves the distinctness of the clustering structure although this structure is assumed to be yet unknown the rationale behind the method is the following i had one known the clusters classes within the data one could facilitate further analysis and reduce space dimensionality by projecting the data to the fisher s linear subspace which by definition best preserves the structure of the given classes ii under some reasonable assumptions this can be done albeit approximately without prior knowledge of the clusters classes in this paper we show how this approach works we present a method of preliminary data transformation that brings the directions of largest overall variability close to the directions of the best between class separation hence for the transformed data simple pca provides an approximation to the fisher s subspace we show that the transformation preserves the distinctness of the unknown structure in the data to a great extent

learning over sets using kernel principal angles we consider the problem of learning with instances defined over a space of sets of vectors we derive a new positive definite kernel f a b defined over pairs of matrices a b based on the concept of principal angles between two linear subspaces we show that the principal angles can be recovered using only inner products between pairs of column vectors of the input matrices thereby allowing the original column vectors of a b to be mapped onto arbitrarily high dimensional feature spaces we demonstrate the usage of the matrix based kernel function f a b with experiments on two visual tasks the first task is the discrimination of irregular motion trajectory of an individual or a group of individuals in a video sequence we use the svm approach using f a b where an input matrix represents the motion trajectory of a group of individuals over a certain fixed time frame we show that the classification irregular versus regular greatly outperforms the conventional representation where all the trajectories form a single vector the second application is the visual recognition of faces from input video sequences representing head motion and facial expressions where f a b is used to compare two image sequences

regularization theory and neural networks architectures we had previously shown that regularization principles lead to approximation schemes that are equivalent to networks with one layer of hidden units called regularization networks in particular s

discriminative learning and recognition of image set classes using canonical correlations we address the problem of comparing sets of images for object recognition where the sets may represent variations in an object s appearance due to changing camera pose and lighting conditions canonical correlations also known as principal or canonical angles which can be thought of as the angles between two d dimensional subspaces have recently attracted attention for image set matching canonical correlations offer many benefits in accuracy efficiency and robustness compared to the two main classical methods parametric distribution based and nonparametric sample based matching of sets here this is first demonstrated experimentally for reasonably sized data sets using existing methods exploiting canonical correlations motivated by their proven effectiveness a novel discriminative learning method over sets is proposed for set classification specifically inspired by classical linear discriminant analysis lda we develop a linear discriminant function that maximizes the canonical correlations of within class sets and minimizes the canonical correlations of between class sets image sets transformed by the discriminant function are then compared by the canonical correlations classical orthogonal subspace method osm is also investigated for the similar purpose and compared with the proposed method the proposed method is evaluated on various object recognition problems using face image sets with arbitrary motion captured under different illuminations and image sets of 500 general objects taken at different views the method is also applied to object category recognition using eth 80 database the proposed method is shown to outperform the state of the art methods in terms of accuracy and efficiency

achieving robust face recognition from video by combining a weak photometric model and a learnt generic face invariant in spite of over two decades of intense research illumination and pose invariance remain prohibitively challenging aspects of face recognition for most practical applications the objective of this work is to recognize faces using video sequences both for training and recognition input in a realistic unconstrained setup in which lighting pose and user motion pattern have a wide variability and face images are of low resolution the central contribution is an illumination invariant which we show to be suitable for recognition from video of loosely constrained head motion in particular there are three contributions i we show how a photometric model of image formation can be combined with a statistical model of generic face appearance variation to exploit the proposed invariant and generalize in the presence of extreme illumination changes ii we introduce a video sequence re illumination algorithm to achieve fine alignment of two video sequences and iii we use the smoothness of geodesically local appearance manifold structure and a robust same identity likelihood to achieve robustness to unseen head poses we describe a fully automatic recognition system based on the proposed method and an extensive evaluation on 323 individuals and 1474 video sequences with extreme illumination pose and head motion variation our system consistently achieved a nearly perfect recognition rate over 99 7 on all four databases

remarks on functional canonical variates alternating least squares methods and ace we discuss properties of some data analytic methods which are intimately related to each other alternating least squares als correspondence analysis and more recently breiman and friedman s ace algorithm the application of these methods to regression produces nonparametric estimators of nonlinear transformations both of the response and the predictors we point out some anomalies as well as some curiosities in the mathematics of these methods and we relate them to some areas in computer aided tomography projection pursuit regression and nonlinear devices in the theory of noise

image set based face recognition using self regularized non negative coding and adaptive distance metric learning simple nearest neighbor classification fails to exploit the additional information in image sets we propose self regularized nonnegative coding to define between set distance for robust face recognition set distance is measured between the nearest set points samples that can be approximated from their orthogonal basis vectors as well as from the set samples under the respective constraints of self regularization and nonnegativity self regularization constrains the orthogonal basis vectors to be similar to the approximated nearest point the nonnegativity constraint ensures that each nearest point is approximated from a positive linear combination of the set samples both constraints are formulated as a single convex optimization problem and the accelerated proximal gradient method with linear time euclidean projection is adapted to efficiently find the optimal nearest points between two image sets using the nearest points between a query set and all the gallery sets as well as the active samples used to approximate them we learn a more discriminative mahalanobis distance for robust face recognition the proposed algorithm works independently of the chosen features and has been tested on gray pixel values and local binary patterns experiments on three standard data sets show that the proposed method consistently outperforms existing state of the art methods

modified cholesky factorizations in interior point algorithms for linear programming we investigate a modified cholesky algorithm typical of those used in most interior point codes for linear programming cholesky based interior point codes are popular for three reasons their implementation requires only minimal changes to standard sparse cholesky algorithms allowing us to take full advantage of software written by specialists in that area they tend to be more efficient than competing approaches that use alternative factorizations and they perform robustly on most practical problems yielding good interior point steps even when the coefficient matrix of the main linear system to be solved for the step components is ill conditioned we investigate this surprisingly robust performance by using analytical tools from matrix perturbation theory and error analysis illustrating our results with computational experiments finally we point out the potential limitations of this approach

discriminant analysis of functional connectivity patterns on grassmann manifold the functional brain networks extracted from fmri images using independent component analysis have been demonstrated informative for distinguishing brain states of cognitive function and brain disorders rather than analyzing each network encoded by a spatial independent component separately we propose a novel algorithm for discriminant analysis of functional brain networks jointly at an individual level the functional brain networks of each individual are used as bases for a linear subspace referred to as a functional connectivity pattern which facilitates a comprehensive characterization of fmri data the functional connectivity patterns of different individuals are analyzed on the grassmann manifold by adopting a principal angle based riemannian distance in conjunction with a support vector machine classifier a forward component selection technique is proposed to select independent components for constructing the most discriminative functional connectivity pattern the discriminant analysis method has been applied to an fmri based schizophrenia study with 31 schizophrenia patients and 31 healthy individuals the experimental results demonstrate that the proposed method not only achieves a promising classification performance for distinguishing schizophrenia patients from healthy controls but also identifies discriminative functional brain networks that are informative for schizophrenia diagnosis c 2011 elsevier inc all rights reserved

a constrained em algorithm for independent component analysis we introduce a novel way of performing independent component analysis using a constrained version of the expectation maximization em algorithm the source distributions are modeled as d one dimensional mixtures of gaussians the observed data are modeled as linear mixtures of the sources with additive isotropic noise this generative model is fit to the data using constrained em the simpler soft switching approach is introduced which uses only one parameter to decide on the sub or supergaussian nature of the sources we explain how our approach relates to independent factor analysis

subspace distance analysis with application to adaptive bayesian algorithm for face recognition we propose subspace distance measures to analyze the similarity between intrapersonal face subspaces which characterize the variations between face images of the same individual we call the conventional intrapersonal subspace average intrapersonal subspace ais because the image differences often come from a large number of persons an intrapersonal subspace is referred to as specific intrapersonal subspace sis if the image differences are from just one person we demonstrate that sis varies significantly from person to person and most siss are not similar to ais based on these observations we introduce the maximum a posteriori map adaptation to the problem of sis estimation and apply it to the bayesian face recognition algorithm experimental results show that the adaptive bayesian algorithm outperforms the non adaptive bayesian algorithm as well as eigenface and fisherface methods if a small number of adaptation images are available

nonextensive information theoretic kernels on measures positive definite kernels on probability measures have been recently applied to classification problems involving text images and other types of structured data some of these kernels are related to classic information theoretic quantities such as shannon s mutual information and the jensen shannon js divergence meanwhile there have been recent advances in nonextensive generalizations of shannon s information theory this paper bridges these two trends by introducing nonextensive information theoretic kernels on probability measures based on new js type divergences these new divergences result from extending the the two building blocks of the classical js divergence convexity and shannon s entropy the notion of convexity is extended to the wider concept of q convexity for which we prove a jensen q inequality based on this inequality we introduce jensen tsallis jt q differences a nonextensive generalization of the js divergence and define a k th order jt q difference between stochastic processes we then define a new family of nonextensive mutual information kernels which allow weights to be assigned to their arguments and which includes the boolean js and linear kernels as particular cases nonextensive string kernels are also defined that generalize the p spectrum kernel we illustrate the performance of these kernels on text categorization tasks in which documents are modeled both as bags of words and as sequences of characters

learning theory for distribution regression we focus on the distribution regression problem regressing to vector valued outputs from probability measures many important machine learning and statistical tasks fit into this framework including multi instance learning and point estimation problems without analytical solution such as hyperparameter or entropy estimation despite the large number of available heuristics in the literature the inherent two stage sampled nature of the problem makes the theoretical analysis quite challenging since in practice only samples from sampled distributions are observable and the estimates have to rely on similarities computed between sets of points to the best of our knowledge the only existing technique with consistency guarantees for distribution regression requires kernel density estimation as an intermediate step which often performs poorly in practice and the domain of the distributions to be compact euclidean in this paper we study a simple analytically computable ridge regression based alternative to distribution regression where we embed the distributions to a reproducing kernel hilbert space and learn the regressor from the embeddings to the outputs our main contribution is to prove that this scheme is consistent in the two stage sampled setup under mild conditions on separable topological domains enriched with kernels we present an exact computational statistical efficiency trade off analysis showing that our estimator is able to match the one stage sampled minimax optimal rate caponnetto and de vito 2007 steinwart et al 2009 this result answers a 17 year old open question establishing the consistency of the classical set kernel haussler 1999 gartner et al 2002 in regression we also cover consistency for more recent kernels on distributions including those due to christmann and steinwart 2010

principal angles separate subject illumination spaces in ydb and cmu pie the theory of illumination subspaces is well developed and has been tested extensively on the yale face database b ydb and cmu pie pie data sets this paper shows that if face recognition under varying illumination is cast as a problem of matching sets of images to sets of images then the minimal principal angle between subspaces is sufficient to perfectly separate matching pairs of image sets from nonmatching pairs of image sets sampled from ydb and pie this is true even for subspaces estimated from as few as six images and when one of the subspaces is estimated from as few as three images if the second subspace is estimated from a larger set 10 or more this suggests that variation under illumination may be thought of as useful discriminating information rather than unwanted noise

dynamic instability of stiffened plates subjected to non uniform harmonic in plane edge loading the dynamic instability characteristics of stiffened plates subjected to in plane partial and concentrated edge loadings are studied using finite element analysis in the structural modelling the plate and the stiffeners are treated as separate elements where the compatibility between these two types of elements is maintained the method of hill s infinite determinants is applied to determine the dynamic instability regions numerical results are presented to study the effects of various parameters such as static load factor aspect ratio boundary conditions stiffening scheme and load parameters on the principal instability regions of stiffened plates using bolotin s method the results show that location size and number of stiffeners have a significant effect on the location of the boundaries of the principal instability region

face recognition using sparse approximated nearest points between image sets we propose an efficient and robust solution for image set classification a joint representation of an image set is proposed which includes the image samples of the set and their affine hull model the model accounts for unseen appearances in the form of affine combinations of sample images to calculate the between set distance we introduce the sparse approximated nearest point sanp sanps are the nearest points of two image sets such that each point can be sparsely approximated by the image samples of its respective set this novel sparse formulation enforces sparsity on the sample coefficients and jointly optimizes the nearest points as well as their sparse approximations unlike standard sparse coding the data to be sparsely approximated are not fixed a convex formulation is proposed to find the optimal sanps between two sets and the accelerated proximal gradient method is adapted to efficiently solve this optimization we also derive the kernel extension of the sanp and propose an algorithm for dynamically tuning the rbf kernel parameter while matching each pair of image sets comprehensive experiments on the ucsd honda cmu mobo and youtube celebrities face datasets show that our method consistently outperforms the state of the art

from sample similarity to ensemble similarity probabilistic distance measures in reproducing kernel hilbert space this paper addresses the problem of characterizing ensemble similarity from sample similarity in a principled manner using a reproducing kernel as a characterization of sample similarity we suggest a probabilistic distance measure in the reproducing kernel hilbert space rkhs as the ensemble similarity assuming normality in the rkhs we derive analytic expressions for probabilistic distance measures that are commonly used in many applications such as chernoff distance or the bhattacharyya distance as its special case kullback leibler divergence etc since the reproducing kernel implicitly embeds a nonlinear mapping our approach presents a new way to study these distances whose feasibility and efficiency is demonstrated using experiments with synthetic and real examples further we extend the ensemble similarity to the reproducing kernel for ensemble and study the ensemble similarity for more general data representations

a fast fixed point algorithm for independent component analysis we introduce a novel fast algorithm for independent component analysis which can be used for blind source separation and feature extraction we show how a neural network learning rule can be transformed into a fixedpoint iteration which provides an algorithm that is very simple does not depend on any user defined parameters and is fast to converge to the most accurate solution allowed by the data the algorithm finds one at a time all nongaussian independent components regardless of their probability distributions the computations can be performed in either batch mode or a semiadaptive manner the convergence of the algorithm is rigorously proved and the convergence speed is shown to be cubic some comparisons to gradient based algorithms are made showing that the new algorithm is usually 10 to 100 times faster sometimes giving the solution in just a few iterations

fuzzy and random sets the two distinct concepts of fuzzy and random sets coexist to form the basics for modeling perception based information gathering processes in artificial intelligence these processes viewed as coarsening schemes provide also justifications for various types of uncertainty measures such as belief functions and possibility measures

twin kernel embedding in most existing dimensionality reduction algorithms the main objective is to preserve relational structure among objects of the input space in a low dimensional embedding space this is achieved by minimizing the inconsistency between two similarity dissimilarity measures one for the input data and the other for the embedded data via a separate matching objective function based on this idea a new dimensionality reduction method called twin kernel embedding tke is proposed tke addresses the problem of visualizing non vectorial data that is difficult for conventional methods in practice due to the lack of efficient vectorial representation tke solves this problem by minimizing the inconsistency between the similarity measures captured respectively by their kernel gram matrices in the two spaces in the implementation by optimizing a nonlinear objective function using the gradient descent algorithm a local minimum can be reached the results obtained include both the optimal similarity preserving embedding and the appropriate values for the hyperparameters of the kernel experimental evaluation on real non vectorial datasets confirmed the effectiveness of tke tke can be applied to other types of data beyond those mentioned in this paper whenever suitable measures of similarity dissimilarity can be defined on the input data

the context tree weighting method basic properties describes a sequential universal data compression procedure for binary tree sources that performs the double mixture using a context tree this method weights in an efficient recursive way the coding distributions corresponding to all bounded memory tree sources and achieves a desirable coding distribution for tree sources with an unknown model and unknown parameters computational and storage complexity of the proposed procedure are both linear in the source sequence length the authors derive a natural upper bound on the cumulative redundancy of the method for individual sequences the three terms in this bound can be identified as coding parameter and model redundancy the bound holds for all source sequence lengths not only for asymptotically large lengths the analysis that leads to this bound is based on standard techniques and turns out to be extremely simple the upper bound on the redundancy shows that the proposed context tree weighting procedure is optimal in the sense that it achieves the rissanen 1984 lower bound

polynomial summaries of positive semidefinite kernels polynomials have proven to be useful tools to tailor generic kernels to specific applications nevertheless we had only restricted knowledge for selecting fertile polynomials which consistently produce positive semidefinite kernels for example the well known polynomial kernel can only take advantage of a very narrow range of polynomials that is the univariate polynomials with positive coefficients this restriction not only hinders intensive exploitation of the flexibility of the kernel method but also causes misuse of indefinite kernels our main theorem significantly relaxes the restriction by asserting that a polynomial consistently produces positive semidefinite kernels if it has a positive semidefinite coefficient matrix this sufficient condition is quite natural and hence it can be a good characterization of the fertile polynomials in fact we prove that the converse of the assertion of the theorem also holds true in the case of degree 1 we also prove the effectiveness of our main theorem by showing three corollaries relating to certain applications known in the literature the first and second corollaries respectively give generalizations of the polynomial kernel and the principal angle determinant kernel the third corollary shows extended and corrected sufficient conditions for the codon improved kernel and the weighted degree kernel with shifts to be positive semidefinite

kernel independent component analysis we present a class of algorithms for independent component analysis ica which use contrast functions based on canonical correlations in a reproducing kernel hilbert space on the one hand we show that our contrast functions are related to mutual information and have desirable mathematical properties as measures of statistical dependence on the other hand building on recent developments in kernel methods we show that these criteria and their derivatives can be computed efficiently minimizing these criteria leads to flexible and robust algorithms for ica we illustrate with simulations involving a wide variety of source distributions showing that our algorithms outperform many of the presently known algorithms

fuzzy sets as a basis for a theory of possibility the theory of possibility described in this paper is related to the theory of fuzzy sets by defining the concept of a possibility distribution as a fuzzy restriction which acts as an elastic constraint on the values that may be assigned to a variable more specifically if f is a fuzzy subset of a universe of discourse u u which is characterized by its membership function f then a proposition of the form x is f where x is a variable taking values in u induces a possibility distribution tx which equates the possibility of x taking the value u to f u the compatibility of u with f in this way x becomes a fuzzy variable which is associated with the possibility distribution tx in much the same way as a random variable is associated with a probability distribution in general a variable may be associated both with a possibility distribution and a probability distribution with the weak connection between the two expressed as the possibility probability consistency principle a thesis advanced in this paper is that the imprecision that is intrinsic in natural languages is in the main possibilistic rather than probabilistic in nature thus by employing the concept of a possibility distribution a proposition p in a natural language may be translated into a procedure which computes the probability distribution of a set of attributes which are implied by p several types of conditional translation rules are discussed and in particular a translation rule for propositions of the form x is f is possible where is a number in the interval 0 1 is formulated and illustrated by examples

natural gradient works efficiently in learning when a parameter space has a certain underlying structure the ordinary gradient of a function does not represent its steepest direction but the natural gradient does information geometry is used for calculating the natural gradients in the parameter space of perceptrons the space of matrices for blind source separation and the space of linear dynamical systems for blind source deconvolution the dynamical behavior of natural gradient online learning is analyzed and is proved to be fisher efficient implying that it has asymptotically the same performance as the optimal batch estimation of parameters this suggests that the plateau phenomenon which appears in the backpropagation learning algorithm of multilayer perceptrons might disappear or might not be so serious when the natural gradient is used an adaptive method of updating the learning rate is proposed and analyzed

boosted manifold principal angles for image set based recognition in this paper we address the problem of classifying vector sets we motivate and introduce a novel method based on comparisons between corresponding vector subspaces in particular there are two main areas of novelty i we extend the concept of principal angles between linear subspaces to manifolds with arbitrary nonlinearities ii it is demonstrated how boosting can be used for application optimal principal angle fusion the strengths of the proposed method are empirically demonstrated on the task of automatic face recognition afr in which it is shown to outperform state of the art methods in the literature

the pyramid match kernel efficient learning with sets of features in numerous domains it is useful to represent a single example by the set of the local features or parts that comprise it however this representation poses a challenge to many conventional machine learning techniques since sets may vary in cardinality and elements lack a meaningful ordering kernel methods can learn complex functions but a kernel over unordered set inputs must somehow solve for correspondences generally a computationally expensive task that becomes impractical for large set sizes we present a new fast kernel function called the pyramid match that measures partial match similarity in time linear in the number of features the pyramid match maps unordered feature sets to multi resolution histograms and computes a weighted histogram intersection in order to find implicit correspondences based on the finest resolution histogram cell where a matched pair first appears we show the pyramid match yields a mercer kernel and we prove bounds on its error relative to the optimal partial matching cost we demonstrate our algorithm on both classification and regression tasks including object recognition 3 d human pose inference and time of publication estimation for documents and we show that the proposed method is accurate and significantly more efficient than current approaches

difference subspace and its generalization for subspace based methods subspace based methods are known to provide a practical solution for image set based object recognition based on the insight that local shape differences between objects offer a sensitive cue for recognition this paper addresses the problem of extracting a subspace representing the difference components between class subspaces generated from each set of object images independently of each other we first introduce the difference subspace ds a novel geometric concept between two subspaces as an extension of a difference vector between two vectors and describe its effectiveness in analyzing shape differences we then generalize it to the generalized difference subspace gds for multi class subspaces and show the benefit of applying this to subspace and mutual subspace methods in terms of recognition capability furthermore we extend these methods to kernel ds kds and kernel gds kgds by a nonlinear kernel mapping to deal with cases involving larger changes in viewing direction in summary the contributions of this paper are as follows 1 a ds kds between two class subspaces characterizes shape differences between the two respectively corresponding objects 2 the projection of an input vector onto a ds kds realizes selective visualization of shape differences between objects and 3 the projection of an input vector or subspace onto a gds kgds is extremely effective at extracting differences between multiple subspaces and therefore improves object recognition performance we demonstrate validity through shape analysis on synthetic and real images of 3d objects as well as extensive comparison of performance on classification tests with several related methods we study the performance in face image classification on the yale face database b and the cmu multi pie database and hand shape classification of multi view images

incremental kernel principal component analysis the kernel principal component analysis kpca has been applied in numerous image related machine learning applications and it has exhibited superior performance over previous approaches such as pca however the standard implementation of kpca scales badly with the problem size making computations for large problems infeasible also the batch nature of the standard kpca computation method does not allow for applications that require online processing this has somewhat restricted the domains in which kpca can potentially be applied this paper introduces an incremental computation algorithm for kpca to address these two problems the basis of the proposed solution lies in computing incremental linear pca in the kernel induced feature space and constructing reduced set expansions to maintain constant update speed and memory usage we also provide experimental results which demonstrate the effectiveness of the approach

gender classification from unaligned facial images using support subspaces rough face alignments result in suboptimal performance of face identification in this study we present an approach for identifying the gender based on facial images without proper face alignments instead of just using only the detected face patch for identification a set of patches is randomly cropped around the face detection region each patch set is represented by a linear subspace and compared with other linear subspaces by measuring their canonical correlations a similarity matrix comprised of the canonical correlations is then incorporated into an indefinite kernel support vector machine svm formulation the number of support vectors which we call support subspaces can be decided automatically hence we can avoid the dimension selection problem observed in our previous work our experimental results demonstrate that the proposed approach outperforms state of the art methods

blind separation of mixture of independent sources through a quasi maximum likelihood approach we propose two methods for separating mixture of independent sources without any precise knowledge of their probability distribution they are obtained by considering a maximum likelihood ml solution corresponding to some given distributions of the sources and relaxing this assumption afterward the first method is specially adapted to temporally independent non gaussian sources and is based on the use of nonlinear separating functions the second method is specially adapted to correlated sources with distinct spectra and is based on the use of linear separating filters a theoretical analysis of the performance of the methods has been made a simple procedure for optimally choosing the separating functions is proposed further in the second method a simple implementation based on the simultaneous diagonalization of two symmetric matrices is provided finally some numerical and simulation results are given illustrating the performance of the method and the good agreement between the experiments and the theory

canonical correlation analysis when the data are curves summary it is not immediately straightforward to extend canonical correlation analysis to the context of functional data analysis where the data are themselves curves or functions the obvious approach breaks down and it is necessary to use a method involving smoothing in some way such a method is introduced and discussed with reference to a data set on human gait the breakdown of the unsmoothed method is illustrated in a practical context and is demonstrated theoretically a consistency theorem for the smoothed method is proved in an increasing number of problems in a wide range of fields the data observed are not the univariate or multivariate observations of classical statistics but are func tions observed continuously ramsay and dalzell 1991 give the name functional data analysis to the analysis of data of this kind in most cases the observations will be functions of time or a closely related variable but there are clearly applications where the functions are surfaces observed over two or three dimensional space the motivating example for the present paper is a set of data on children s gait collected by the motion analysis laboratory at children s hospital san diego california see olshen et al 1989 for full details for each of a number of children several angles made by the child s joints knee hip etc are observed during the child s gait cycle one aim of the study is to gain understanding of the gait cycle of a normal child to make comparisons with children suffering from walking diffi culties these data motivated rice and silverman 1991 to discuss the extension of principal component analysis to the functional setting and to explain how smoothing can be incorporated into the analysis in a natural way in a substantial complemen tary paper ramsay and dalzell 1991 discuss different approaches to regression and principal component analysis in functional data analysis illustrating their work by a meteorological example the insights of functional data analysis may be helpful in chemometrics for instance a standard chemometric problem is the analysis of spectra observed in chromatography and these are to all intents and purposes functional observations another obvious area of potential relevance is the analysis of growth curve data the focus in the present paper will be on canonical correlation analysis cca in the gait example one might ask how variability in the knee angle cycle is related to

geodesic distance on a grassmannian for monitoring the progression of alzheimer s disease we propose a geodesic distance on a grassmannian manifold that can be used to quantify the shape progression patterns of the bilateral hippocampi amygdalas and lateral ventricles in healthy control hc mild cognitive impairment mci and alzheimer s disease ad longitudinal magnetic resonance imaging mri scans of 754 subjects 3092 scans in total were used in this study longitudinally the geodesic distance was found to be proportional to the elapsed time separating the two scans in question cross sectionally utilizing a linear mixed effects statistical model we found that each structure s annualized rate of change in the geodesic distance followed the order of ad mci hc with statistical significance being reached in every case in addition for each of the six structures of interest within the same time interval e g from baseline to the 6th month we observed significant correlations between the geodesic distance and the cognitive deterioration as quantified by the adas cog increase and the mmse decrease furthermore as the disease progresses over time this linkage between the inter shape geodesic distance and the cognitive decline becomes considerably stronger and more significant

subspace learning for unsupervised feature selection via matrix factorization dimensionality reduction is an important and challenging task in machine learning and data mining feature selection and feature extraction are two commonly used techniques for decreasing dimensionality of the data and increasing efficiency of learning algorithms specifically feature selection realized in the absence of class labels namely unsupervised feature selection is challenging and interesting in this paper we propose a new unsupervised feature selection criterion developed from the viewpoint of subspace learning which is treated as a matrix factorization problem the advantages of this work are four fold first dwelling on the technique of matrix factorization a unified framework is established for feature selection feature extraction and clustering second an iterative update algorithm is provided via matrix factorization which is an efficient technique to deal with high dimensional data third an effective method for feature selection with numeric data is put forward instead of drawing support from the discretization process fourth this new criterion provides a sound foundation for embedding kernel tricks into feature selection with this regard an algorithm based on kernel methods is also proposed the algorithms are compared with four state of the art feature selection methods using six publicly available datasets experimental results demonstrate that in terms of clustering results the proposed two algorithms come with better performance than the others for almost all datasets we experimented with here highlightspropose a new feature selection based on matrix factorization present a fast convergent algorithm for matrix factorization on certain constraints incorporate kernel tricks into feature selection problems construct a unified framework for feature extraction feature selection and clustering

semigroup kernels on measures we present a family of positive definite kernels on measures characterized by the fact that the value of the kernel between two measures is a function of their sum these kernels can be used to derive kernels on structured objects such as images and texts by representing these objects as sets of components such as pixels or words or more generally as measures on the space of components several kernels studied in this work make use of common quantities defined on measures such as entropy or generalized variance to detect similarities given an a priori kernel on the space of components itself the approach is further extended by restating the previous results in a more efficient and flexible framework using the kernel trick finally a constructive approach to such positive definite kernels through an integral representation theorem is proved before presenting experimental results on a benchmark experiment of handwritten digits classification to illustrate the validity of the approach

infinite dimensional log determinant divergences between positive definite trace class operators abstract this work presents a novel parametrized family of log determinant log det divergences between positive definite unitized trace class operators on a hilbert space this is a generalization of the log det divergences between symmetric positive definite matrices to the infinite dimensional setting for the log det divergences between covariance operators on a reproducing kernel hilbert space rkhs we obtain closed form solutions via the corresponding gram matrices by employing the log det divergences we then generalize the bhattacharyya and hellinger distances and the kullback leibler and renyi divergences between multivariate normal distributions to gaussian measures on an infinite dimensional hilbert space

manifold manifold distance and its application to face recognition with image sets in this paper we address the problem of classifying image sets for face recognition where each set contains images belonging to the same subject and typically covering large variations by modeling each image set as a manifold we formulate the problem as the computation of the distance between two manifolds called manifold manifold distance mmd since an image set can come in three pattern levels point subspace and manifold we systematically study the distance among the three levels and formulate them in a general multilevel mmd framework specifically we express a manifold by a collection of local linear models each depicted by a subspace mmd is then converted to integrate the distances between pairs of subspaces from one of the involved manifolds we theoretically and experimentally study several configurations of the ingredients of mmd the proposed method is applied to the task of face recognition with image sets where identification is achieved by seeking the minimum mmd from the probe to the gallery of image sets our experiments demonstrate that as a general set similarity measure mmd consistently outperforms other competing nondiscriminative methods and is also promisingly comparable to the state of the art discriminative methods

a new metric for probability distributions we introduce a metric for probability distributions which is bounded information theoretically motivated and has a natural bayesian interpretation the square root of the well known spl chi sup 2 distance is an asymptotic approximation to it moreover it is a close relative of the capacitory discrimination and jensen shannon divergence

class incremental generalized discriminant analysis generalized discriminant analysis gda is the nonlinear extension of the classical linear discriminant analysis lda via the kernel trick mathematically gda aims to solve a generalized eigenequation problem which is always implemented by the use of singular value decomposition svd in the previously proposed gda algorithms a major drawback of svd however is the difficulty of designing an incremental solution for the eigenvalue problem moreover there are still numerical problems of computing the eigenvalue problem of large matrices in this article we propose another algorithm for solving gda as for the case of small sample size problem which applies qr decomposition rather than svd a major contribution of the proposed algorithm is that it can incrementally update the discriminant vectors when new classes are inserted into the training set the other major contribution of this article is the presentation of the modified kernel gram schmidt mkgs orthogonalization algorithm for implementing the qr decomposition in the feature space which is more numerically stable than the kernel gram schmidt kgs algorithm we conduct experiments on both simulated and real data to demonstrate the better performance of the proposed methods

an information maximization approach to blind separation and blind deconvolution we derive a new self organizing learning algorithm that maximizes the information transferred in a network of nonlinear units the algorithm does not assume any knowledge of the input distributions and is defined here for the zero noise limit under these conditions information maximization has extra properties not found in the linear case linsker 1989 the nonlinearities in the transfer function are able to pick up higher order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation this enables the network to separate statistically independent components in the inputs a higher order generalization of principal components analysis we apply the network to the source separation or cocktail party problem successfully separating unknown mixtures of up to 10 speakers we also show that a variant on the network architecture is able to perform blind deconvolution cancellation of unknown echoes and reverberation in a speech signal finally we derive dependencies of information transfer on time delays we suggest that information maximization provides a unifying framework for problems in blind signal processing