bidual renormings of banach spaces abstract given a separable banach space x with no isomorphic copies of l 1 and a separable subspace y of its bidual we provide a sufficient condition on y to ensure that x admits an equivalent norm such that the restriction to y of the corresponding bidual norm is midpoint locally uniformly rotund this result applies to the separable subspaces of the bidual of a banach space with a shrinking unconditional schauder basis and to the bidual of the james space

learning overcomplete representations in an overcomplete basis the number of basis vectors is greater than the dimensionality of the input and the representation of an input is not a unique combination of basis vectors overcomplete representations have been advocated because they have greater robustness in the presence of noise can be sparser and can have greater flexibility in matching structure in the data overcomplete codes have also been proposed as a model of some of the response properties of neurons in primary visual cortex previous work has focused on finding the best representation of a signal using a fixed overcomplete basis or dictionary we present an algorithm for learning an overcomplete basis by viewing it as probabilistic model of the observed data we show that overcomplete bases can yield a better approximation of the underlying statistical distribution of the data and can thus lead to greater coding efficiency this can be viewed as a generalization of the technique of independent component analysis and provides a method for bayesian reconstruction of signals in the presence of noise and for blind source separation when there are more sources than mixtures

on the selection and classification of independent features this paper is focused on the problems of feature selection and classification when classes are modeled by statistically independent features we show that under the assumption of class conditional independence the class separability measure of divergence is greatly simplified becoming a sum of unidimensional divergences providing a feature selection criterion where no exhaustive search is required since the hypothesis of independence is infrequently met in practice we also provide a framework making use of class conditional independent component analyzers where this assumption can be held on stronger grounds divergence and the bayes decision scheme are adapted to this class conditional representation an algorithm that integrates the proposed representation feature selection technique and classifier is presented experiments on artificial benchmark and real world data illustrate our technique and evaluate its performance

on the spectral radius of trees with fixed diameter let t n d be the set of trees on n vertices with diameter d in this paper the first d2 1 spectral radii of trees in the set t n d 3 d n 4 are characterized

eigenfaces vs fisherfaces recognition using class specific linear projection we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression taking a pattern classification approach we consider each pixel in an image as a coordinate in a high dimensional space we take advantage of the observation that the images of a particular face under varying illumination but fixed pose lie in a 3d linear subspace of the high dimensional image space if the face is a lambertian surface without shadowing however since faces are not truly lambertian surfaces and do indeed produce self shadowing images will deviate from this linear subspace rather than explicitly modeling this deviation we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation our projection method is based on fisher s linear discriminant and produces well separated classes in a low dimensional subspace even under severe variation in lighting and facial expressions the eigenface technique another method based on linearly projecting the image space to a low dimensional subspace has similar computational requirements yet extensive experimental results demonstrate that the proposed fisherface method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases

modified principal component analysis an integration of multiple similarity subspace models we modify the conventional principal component analysis pca and propose a novel subspace learning framework modified pca mpca using multiple similarity measurements mpca computes three similarity matrices exploiting the similarity measurements 1 mutual information 2 angle information and 3 gaussian kernel similarity we employ the eigenvectors of similarity matrices to produce new subspaces referred to as similarity subspaces a new integrated similarity subspace is then generated using a novel feature selection approach this approach needs to construct a kind of vector set termed weak machine cell wmc which contains an appropriate number of the eigenvectors spanning the similarity subspaces combining the wrapper method and the forward selection scheme mpca selects a wmc at a time that has a powerful discriminative capability to classify samples mpca is very suitable for the application scenarios in which the number of the training samples is less than the data dimensionality mpca outperforms the other state of the art pca based methods in terms of both classification accuracy and clustering result in addition mpca can be applied to face image reconstruction mpca can use other types of similarity measurements extensive experiments on many popular real world data sets such as face databases show that mpca achieves desirable classification results as well as has a powerful capability to represent data

probabilistic visual learning for object representation we present an unsupervised technique for visual learning which is based on density estimation in high dimensional spaces using an eigenspace decomposition two types of density estimates are derived for modeling the training data a multivariate gaussian for unimodal distributions and a mixture of gaussians model for multimodal distributions those probability densities are then used to formulate a maximum likelihood estimation framework for visual search and target detection for automatic object recognition and coding our learning technique is applied to the probabilistic visual modeling detection recognition and coding of human faces and nonrigid objects such as hands

algebraic feature extraction for image recognition based on an optimal discriminant criterion abstract a novel algebraic feature extraction method for image recognition is presented the main difference between the present method and any other existing algebraic feature extraction method is that the present method uses an optimal discriminant criterion to extract algebraic features for the given training image samples a set of optimal discriminant projection vectors is calculated according to a generalized fisher criterion function on the basis of the optimal discriminant projection vector the projective feature vectors of an image i e algebraic features are extracted by projecting the image onto all optimal discriminant projection vectors the method of calculating the set of optimal discriminant projection vectors is discussed a minimum distance classifier is also proposed for classifying projective feature vectors experimental results showed that the present method has good recognition performance an important conclusion about the present method is that the foley sammon optimal set of discriminant vectors is a special case of the set of optimal discriminant projection vectors

multilinear supervised neighborhood embedding of a local descriptor tensor for scene object recognition in this paper we propose to represent an image as a local descriptor tensor and use a multilinear supervised neighborhood embedding msne for discriminant feature extraction which is able to be used for subject or scene recognition the contributions of this paper include 1 a novel feature extraction approach denoted as the histogram of orientation weighted with a normalized gradient nhog for local region representation which is robust to large illumination variation in an image 2 an image representation framework denoted as the local descriptor tensor which can effectively combine a moderate amount of local features together for image representation and be more efficient than the popular existing bag of feature model and 3 an msne analysis algorithm which can directly deal with the local descriptor tensor for extracting discriminant and compact features and at the same time preserve neighborhood structure in tensor feature space for subject scene recognition we demonstrate the performance advantages of our proposed approach over existing techniques on different types of benchmark database such as a scene data set i e ot8 face data sets i e yale and pie and view based object data sets coil 100 and eth 80

multiscale patch based contrast measure for small infrared target detection infrared ir small target detection plays an important role in ir guidance systems in this paper a biologically inspired method called multiscale patch based contrast measure mpcm is proposed for small ir target detection mpcm can increase the contrast between target and background which makes it easy to segment small target by simple adaptive thresholding method experimental results on three sequences demonstrate that the proposed small target detection method can not only suppress background clutter effectively even if with strong noise interference but also detect targets accurately with low false alarm rate and high speed highlightsa biologically inspired target enhancement method called multiscale patch based contrast measure mpcm is presented based on mpcm a small ir target detection algorithm is designed the proposed small target detection method achieves promising detection performance on three real ir image sequences

an alternative perspective on adaptive independent component analysis algorithms this article develops an extended independent component analysis algorithm for mixtures of arbitrary subgaussian and supergaussian sources the gaussian mixture model of pearson is employed in deriving a closedform generic score function for strictly subgaussian sources this is combined with the score function for a unimodal supergaussian density to provide a computationally simple yet powerful algorithm for performing independent component analysis on arbitrary mixtures of nongaussian sources

exploring latent structure of mixture ica models by the minimum β divergence method independent component analysis ica attempts to extract original independent signals source components that are linearly mixed in a basic framework this letter discusses a learning algorithm for the separation of different source classes in which the observed data follow a mixture of several ica models where each model is described by a linear combination of independent and nongaussian sources the proposed method is based on a sequential application of the minimum divergence method to separate all source classes sequentially the proposed method searches the recovering matrix of each class on the basis of a rule of sequential change of the shifting parameter if the initial choice of the shifting parameter vector is close to the mean of a data class then all of the hidden sources belonging to that class are recovered properly with independent and nongaussian structure considering the data in other classes as outliers the value of the tuning parameter is a key in the performance of the proposed method a cross validation technique is proposed as an adaptive selection procedure for the tuning parameter for this algorithm together with applications for both real and synthetic data analysis

multi linear neighborhood preserving projection for face recognition this paper proposes a novel method of supervised and unsupervised multi linear neighborhood preserving projection mnpp for face recognition unlike conventional neighborhood preserving projections the mnpp method operates directly on tensorial data rather than vectors or matrices and solves problems of tensorial representation for multi dimensional feature extraction classification and recognition as opposed to traditional approaches such as npp and 2dnpp which derive only one subspace multiple interrelated subspaces are obtained in the mnpp method by unfolding the tensor over different tensorial directions the number of subspaces derived by mnpp is determined by the order of the tensor space this approach is used for face recognition and biometrical security classification problems involving higher order tensors the performance of our proposed and existing techniques is analyzed using three benchmark facial datasets orl ar and feret the obtained results show that the mnpp outperforms the standard approaches in terms of the error rate a new method for supervised and unsupervised dimensionality reduction problem work in embedded manifold to allow representation of multi dimensional images natural extension to 2 d 3 d or higher order tensors combine intrinsic geometry and discriminative information for face recognition avoid the small sample size and the curse of dimensionality problems

a regularized approach for geodesic based semisupervised multimanifold learning geodesic distance as an essential measurement for data dissimilarity has been successfully used in manifold learning however most geodesic distance based manifold learning algorithms have two limitations when applied to classification 1 class information is rarely used in computing the geodesic distances between data points on manifolds and 2 little attention has been paid to building an explicit dimension reduction mapping for extracting the discriminative information hidden in the geodesic distances in this paper we regard geodesic distance as a kind of kernel which maps data from linearly inseparable space to linear separable distance space in doing this a new semisupervised manifold learning algorithm namely regularized geodesic feature learning algorithm is proposed the method consists of three techniques a semisupervised graph construction method replacement of original data points with feature vectors which are built by geodesic distances and a new semisupervised dimension reduction method for feature vectors experiments on the mnist usps handwritten digit data sets mit cbcl face versus nonface data set and an intelligent traffic data set show the effectiveness of the proposed algorithm

the ordering of trees and connected graphs by algebraic connectivity abstract in this paper we first determine that the first four trees of order n 9 with the smallest algebraic connectivity are p n q n w n and z n with p n q n w n z n t where t is any tree of order n other than p n q n w n and z n then we consider the effect on the laplacian eigenvalues of connected graphs by suitably adding edges by using these results and methods we finally determine that the first six connected graphs of order n 9 with the smallest algebraic connectivity are p n q n q n w n w n and w n with p n q n q n w n w n w n g where g is any connected graph of order n other than p n q n q n w n w n and w n

the matched subspace detector with interaction effects propose a new hyperspectral target detection method called msdinter introduce interaction effects into the popular matched subspace detector msd model the target background interaction in hypothesis testing demonstrate superior performance of msdinter on hyperspectral imaging datasets this paper aims to propose a new hyperspectral target detection method termed the matched subspace detector with interaction effects msdinter the msdinter introduces interaction effects terms into the popular matched subspace detector msd from regression analysis in multivariate statistics and the bilinear mixing model in hyperspectral unmixing in this way the interaction between the target and the surrounding background which should have but not yet been considered by the msd is modelled and estimated such that superior performance of target detection can be achieved besides deriving the msdinter methodologically we also demonstrate its superiority empirically using two hyperspectral imaging datasets

a low rank approximation based transductive support tensor machine for semisupervised classification in the fields of machine learning pattern recognition image processing and computer vision the data are usually represented by the tensors for the semisupervised tensor classification the existing transductive support tensor machine tstm needs to resort to iterative technique which is very time consuming in order to overcome this shortcoming in this paper we extend the concave convex procedure based transductive support vector machine cccp tsvm to the tensor patterns and propose a low rank approximation based tstm in which the tensor rank one decomposition is used to compute the inner product of the tensors theoretically concave convex procedure based tstm cccp tstm is an extension of the linear cccp tsvm to tensor patterns when the input patterns are vectors cccp tstm degenerates into the linear cccp tsvm a set of experiments is conducted on 23 semisupervised classification tasks which are generated from seven second order face data sets three third order gait data sets and two third order image data sets to illustrate the performance of the cccp tstm the results show that compared with cccp tsvm and tstm cccp tstm provides significant performance gain in terms of test accuracy and training speed

mercer kernel based clustering in feature space the article presents a method for both the unsupervised partitioning of a sample of data and the estimation of the possible number of inherent clusters which generate the data this work exploits the notion that performing a nonlinear data transformation into some high dimensional feature space increases the probability of the linear separability of the patterns within the transformed space and therefore simplifies the associated data structure it is shown that the eigenvectors of a kernel matrix which defines the implicit mapping provides a means to estimate the number of clusters inherent within the data and a computationally simple iterative procedure is presented for the subsequent feature space partitioning of the data

blind source separation semiparametric statistical approach the semiparametric statistical model is used to formulate the problem of blind source separation the method of estimating functions is applied to this problem it is shown that an estimator of the mixing matrix or its learning version can be described in terms of an estimating function the statistical efficiencies of these algorithms are studied the main results are as follows 1 the space consisting of all the estimating functions is derived 2 the space is decomposed into the orthogonal sum of the admissible part and a redundant ancillary part for any estimating function one can find a better or equally good estimator in the admissible part 3 the fisher efficient that is asymptotically best estimating functions are derived 4 the stability of learning algorithms is studied

projection pursuit mixture density estimation in this paper we seek a gaussian mixture model gmm of an n variate probability density function usually the parameters of gmms are determined in the original n dimensional space by optimizing a maximum likelihood ml criterion a practical deficiency of this method of fitting gmms is its poor performance when dealing with high dimensional data since a large sample size is needed to match the accuracy that is possible in low dimensions we propose a method for fitting the gmm based on the projection pursuit strategy this gmm is highly constrained and hence its ability to model structure in subspaces is enhanced compared to a direct ml fitting of a gmm in high dimensions our method is closely related to recently developed independent factor analysis ifa mixture models the comparisons with ml fitting of gmm in n dimensions and ifa mixtures show that the proposed method is an attractive choice for fitting gmms using small sizes of training sets

ica mixture models for unsupervised classification of non gaussian classes and automatic context switching in blind signal separation an unsupervised classification algorithm is derived by modeling observed data as a mixture of several mutually exclusive classes that are each described by linear combinations of independent non gaussian densities the algorithm estimates the density of each class and is able to model class distributions with non gaussian structure the new algorithm can improve classification accuracy compared with standard gaussian mixture models when applied to blind source separation in nonstationary environments the method can switch automatically between classes which correspond to contexts with different mixing properties the algorithm can learn efficient codes for images containing both natural scenes and text this method shows promise for modeling non gaussian structure in high dimensional data and has many potential applications

ica in signals with multiplicative noise independent component analysis ica has been shown in the last few years to be a very useful tool in blind separation of sources and feature extraction however at least in its simpler form its utility is reduced to the case that the outputs are linear mixtures of independent sources this excludes signals with multiplicative noise in this paper ica is extended to this situation in order to do this the special structure that appears in this new model is first studied and then the multiplicative ica method is designed such that it uses this structure to find the mixture of the sources in the noisy environment the local and global convergence properties of the method are also studied and its performance compared with standard ica methods

unsupervised image classification segmentation and enhancement using ica mixture models an unsupervised classification algorithm is derived by modeling observed data as a mixture of several mutually exclusive classes that are each described by linear combinations of independent non gaussian densities the algorithm estimates the data density in each class by using parametric nonlinear functions that fit to the non gaussian structure of the data this improves classification accuracy compared with standard gaussian mixture models when applied to images the algorithm can learn efficient codes basis functions for images that capture the statistically significant structure intrinsic in the images we apply this technique to the problem of unsupervised classification segmentation and denoising of images we demonstrate that this method was effective in classifying complex image textures such as natural scenes and text it was also useful for denoising and filling in missing pixels in images with complex structures the advantage of this model is that image codes can be learned with increasing numbers of classes thus providing greater flexibility in modeling structure and in finding more image features than in either gaussian mixture models or standard independent component analysis ica algorithms

patch alignment manifold matting image matting is generally modeled as a space transform from the color space to the alpha space by estimating the alpha factor of the model the foreground of an image can be extracted however there is some dimensional information redundancy in the alpha space it usually leads to the misjudgments of some pixels near the boundary between the foreground and the background in this paper a manifold matting framework named patch alignment manifold matting is proposed for image matting in particular we first propose a part modeling of color space in the local image patch we then perform whole alignment optimization for approximating the alpha results using subspace reconstructing error furthermore we utilize nesterov s algorithm to solve the optimization problem finally we apply some manifold learning methods in the framework and obtain several image matting methods such as named isomap matting and its derived cascade isomap matting the experimental results reveal that the manifold matting framework and its two examples are effective when compared with several representative matting methods

a unifying information theoretic framework for independent component analysis abstract we show that different theories recently proposed for independent component analysis ica lead to the same iterative learning algorithm for blind separation of mixed independent sources we review those theories and suggest that information theory can be used to unify several lines of research pearlmutter and parra 1 and cardoso 2 showed that the infomax approach of bell and sejnowski 3 and the maximum likelihood estimation approach are equivalent we show that negentropy maximization also has equivalent properties and therefore all three approaches yield the same learning rule for a fixed nonlinearity girolami and fyfe 4 have shown that the nonlinear principal component analysis pca algorithm of karhunen and joutsensalo 5 and oja 6 can also be viewed from information theoretic principles since it minimizes the sum of squares of the fourth order marginal cumulants and therefore approximately minimizes the mutual information 7 lambert 8 has proposed different bussgang cost functions for multichannel blind deconvolution we show how the bussgang property relates to the infomax principle finally we discuss convergence and stability as well as future research issues in blind source separation

real time constrained linear discriminant analysis to target detection and classification in hyperspectral imagery in this paper we present a constrained linear discriminant analysis clda approach to hyperspectral image detection and classification as well as its real time implementation the basic idea of clda is to design an optimal transformation matrix which can maximize the ratio of inter class distance to intra class distance while imposing the constraint that different class centers after transformation are along different directions such that different classes can be better separated the solution turns out to be a constrained version of orthogonal subspace projection osp implemented with a data whitening process the clda approach can be applied to solve both detection and classification problems in particular by introducing color for display the classification is achieved with a single classified image where a pre assigned color is used to display a specified class the real time implementation is also developed to meet the requirement of on line image analysis when the immediate data assessment is critical the experiments using hydice data demonstrate the strength of clda approach in discriminating the targets with subtle spectral difference

a fast fixed point algorithm for independent component analysis we introduce a novel fast algorithm for independent component analysis which can be used for blind source separation and feature extraction we show how a neural network learning rule can be transformed into a fixedpoint iteration which provides an algorithm that is very simple does not depend on any user defined parameters and is fast to converge to the most accurate solution allowed by the data the algorithm finds one at a time all nongaussian independent components regardless of their probability distributions the computations can be performed in either batch mode or a semiadaptive manner the convergence of the algorithm is rigorously proved and the convergence speed is shown to be cubic some comparisons to gradient based algorithms are made showing that the new algorithm is usually 10 to 100 times faster sometimes giving the solution in just a few iterations

the cfar adaptive subspace detector is a scale invariant glrt the constant false alarm rate cfar matched subspace detector cfar msd is the uniformly most powerful invariant test and the generalized likelihood ratio test glrt for detecting a target signal in noise whose covariance structure is known but whose level is unknown previously the cfar adaptive subspace detector cfar asd or adaptive coherence estimator ace was proposed for detecting a target signal in noise whose covariance structure and level are both unknown and whose covariance structure is estimated with a sample covariance matrix based on training data we show here that the cfar asd is glrt when the test measurement is not constrained to have the same noise level as the training data as a consequence this glrt is invariant to a more general scaling condition on the test and training data than the well known glrt of kelly 1986

a general procedure for learning mixtures of independent component analyzers this paper presents a new procedure for learning mixtures of independent component analyzers the procedure includes non parametric estimation of the source densities supervised unsupervised learning of the model parameters incorporation of any independent component analysis ica algorithm into the learning of the ica mixtures and estimation of residual dependencies after training for correction of the posterior probability of every class to the testing observation vector we demonstrate the performance of the procedure in the classification of ica mixtures of two three and four classes of synthetic data and in the classification of defective materials consisting of 3d finite element models and lab specimens in non destructive testing using the impact echo technique the application of the proposed posterior probability correction demonstrates an improvement in the classification accuracy semi supervised learning shows that unlabeled data can degrade the performance of the classifier when they do not fit the generative model comparative results of the proposed method and standard ica algorithms for blind source separation in one and multiple ica data mixtures show the suitability of the non parametric ica mixture based method for data modeling

target detection based on a dynamic subspace for hyperspectral target detection it is usually the case that only part of the targets pixels can be used as target signatures so can we use them to construct the most proper background subspace for detecting all the probable targets in this paper a dynamic subspace detection dsd method which establishes a multiple detection framework is proposed in each detection procedure blocks of pixels are calculated by the random selection and the succeeding detection performance distribution analysis manifold analysis is further used to eliminate the probable anomalous pixels and purify the subspace datasets and the remaining pixels construct the subspace for each detection procedure the final detection results are then enhanced by the fusion of target occurrence frequencies in all the detection procedures experiments with both synthetic and real hyperspectral images hsi evaluate the validation of our proposed dsd method by using several different state of the art methods as the basic detectors with several other single detectors and multiple detection methods as comparable methods improved receiver operating characteristic curves and better separability between targets and backgrounds by the dsd methods are illustrated the dsd methods also perform well with the covariance based detectors showing their efficiency in selecting covariance information for detection the dynamic subspace detection theory is a useful background statistics estimation our method can find the most suitable pixels to construct detectors targets can be further distinguished by a fusion of all the detection procedures

matched shrunken cone detector mscd bayesian derivations and case studies for hyperspectral target detection hyperspectral images hsis possess non negative properties for both hyperspectral signatures and abundance coefficients which can be naturally modeled using cone based representation however in hyperspectral target detection cone based methods are barely studied in this paper we propose a new regularized cone based representation approach to hyperspectral target detection as well as its two working models by incorporating into the cone representation l 2 norm and l 1 norm regularizations respectively we call the new approach the matched shrunken cone detector mscd also important we provide principled derivations of the proposed mscd from the bayesian perspective we show that mscd can be derived by assuming a multivariate half gaussian distribution or a multivariate half laplace distribution as the prior distribution of the coefficients of the models in the experimental studies we compare the proposed mscd with the subspace methods and the sparse representation based methods for hsi target detection two real hyperspectral data sets are used for evaluating the detection performances on sub pixel targets and full pixel targets respectively results show that the proposed mscd can outperform other methods in both cases demonstrating the competitiveness of the regularized cone based representation

multilinear discriminant analysis for higher order tensor data classification in the past decade great efforts have been made to extend linear discriminant analysis for higher order data classification generally referred to as multilinear discriminant analysis mda existing examples include general tensor discriminant analysis gtda and discriminant analysis with tensor representation dater both the two methods attempt to resolve the problem of tensor mode dependency by iterative approximation gtda is known to be the first mda method that converges over iterations however its performance relies highly on the tuning of the parameter in the scatter difference criterion although dater usually results in better classification performance it does not converge yet the number of iterations executed has a direct impact on dater s performance in this paper we propose a closed form solution to the scatter difference objective in gtda namely direct gtda dgtda which also gets rid of parameter tuning we demonstrate that dgtda outperforms gtda in terms of both efficiency and accuracy in addition we propose constrained multilinear discriminant analysis cmda that learns the optimal tensor subspace by iteratively maximizing the scatter ratio criterion we prove both theoretically and experimentally that the value of the scatter ratio criterion in cmda approaches its extreme value if it exists with bounded error leading to superior and more stable performance in comparison to dater

natural gradient works efficiently in learning when a parameter space has a certain underlying structure the ordinary gradient of a function does not represent its steepest direction but the natural gradient does information geometry is used for calculating the natural gradients in the parameter space of perceptrons the space of matrices for blind source separation and the space of linear dynamical systems for blind source deconvolution the dynamical behavior of natural gradient online learning is analyzed and is proved to be fisher efficient implying that it has asymptotically the same performance as the optimal batch estimation of parameters this suggests that the plateau phenomenon which appears in the backpropagation learning algorithm of multilayer perceptrons might disappear or might not be so serious when the natural gradient is used an adaptive method of updating the learning rate is proposed and analyzed

a survey of multilinear subspace learning for tensor data increasingly large amount of multidimensional data are being generated on a daily basis in many applications this leads to a strong demand for learning algorithms to extract useful information from these massive data this paper surveys the field of multilinear subspace learning msl for dimensionality reduction of multidimensional data directly from their tensorial representations it discusses the central issues of msl including establishing the foundations of the field via multilinear projections formulating a unifying msl framework for systematic treatment of the problem examining the algorithmic aspects of typical msl solutions and categorizing both unsupervised and supervised msl algorithms into taxonomies lastly the paper summarizes a wide range of msl applications and concludes with perspectives on future research directions

unsupervised real time constrained linear discriminant analysis to hyperspectral image classification we have proposed a constrained linear discriminant analysis clda approach for classifying the remotely sensed hyperspectral images its basic idea is to design an optimal linear transformation operator which can maximize the ratio of inter class to intra class distance while satisfying the constraint that the different class centers after transformation are aligned along different directions its major advantage over the traditional fisher s linear discriminant analysis is that the classification can be achieved simultaneously with the transformation the clda is a supervised approach i e the class spectral signatures need to be known a priori but in practice these informations may be difficult or even impossible to obtain so in this paper we will extend the clda algorithm into an unsupervised version where the class spectral signatures are to be directly generated from an unknown image scene computer simulation is used to evaluate how well the algorithm performs in terms of finding the pure signatures we will also discuss how to implement the unsupervised clda algorithm in real time for resolving the critical situations when the immediate data analysis results are required

multiple rank multi linear svm for matrix data classification matrices or more generally multi way arrays tensors are common forms of data that are encountered in a wide range of real applications how to classify this kind of data is an important research topic for both pattern recognition and machine learning in this paper by analyzing the relationship between two famous traditional classification approaches i e svm and stm a novel tensor based method i e multiple rank multi linear svm mrmlsvm is proposed different from traditional vector based and tensor based methods multiple rank left and right projecting vectors are employed to construct decision boundary and establish margin function we reveal that the rank of transformation can be regarded as a tradeoff parameter to balance the capacity of learning and generalization in essence we also proposed an effective approach to solve the proposed non convex optimization problem the convergence behavior initialization computational complexity and parameter determination problems are analyzed compared with vector based classification methods mrmlsvm achieves higher accuracy and has lower computational complexity compared with traditional supervised tensor based methods mrmlsvm performs better for matrix data classification promising experimental results on various kinds of data sets are provided to show the effectiveness of our method highlightswe have proposed a novel method i e mrmlsvm for matrix data classification we have revealed the essence of mrmlsvm from the view of learning theory we have provided an effective way to solve the proposed non convex problem our proposed approach is a common model and mrmlsvm is just an instance

a hierarchical bayesian model for learning nonlinear statistical regularities in nonstationary natural signals capturing statistical regularities in complex high dimensional data is an important problem in machine learning and signal processing models such as principal component analysis pca and independent component analysis ica make few assumptions about the structure in the data and have good scaling properties but they are limited to representing linear statistical regularities and assume that the distribution of the data is stationary for many natural complex signals the latent variables often exhibit residual dependencies as well as nonstationary statistics here we present a hierarchical bayesian model that is able to capture higher order nonlinear structure and represent nonstationary data distributions the model is a generalization of ica in which the basis function coefficients are no longer assumed to be independent instead the dependencies in their magnitudes are captured by a set of density components each density component describes a common pattern of deviation from the marginal density of the pattern ensemble in different combinations they can describe nonstationary distributions adapting the model to image or audio data yields a nonlinear distributed code for higher order statistical regularities that reflect more abstract invariant properties of the signal

on the spectral moments of unicyclic graphs with fixed diameter abstract unicyclic graphs are connected graphs in which the number of edges equals the number of vertices let u n d be the class of unicyclic graphs of order n and diameter d for unicyclic graphs lexicographic ordering by spectral moments s order is discussed in this paper the last d d 2 2 graphs in the s order among all unicyclic graphs in u n d 3 d n 5 are characterized in addition for the cases d n 4 and d n 3 the last 2 n 11 and last n n 2 7 unicyclic graphs in the set u n d are also given respectively

geometric mean for subspace selection subspace selection approaches are powerful tools in pattern classification and data visualization one of the most important subspace approaches is the linear dimensionality reduction step in the fisher s linear discriminant analysis flda which has been successfully employed in many fields such as biometrics bioinformatics and multimedia information management however the linear dimensionality reduction step in flda has a critical drawback for a classification task with c classes if the dimension of the projected subspace is strictly lower than c 1 the projection to a subspace tends to merge those classes which are close together in the original feature space if separate classes are sampled from gaussian distributions all with identical covariance matrices then the linear dimensionality reduction step in flda maximizes the mean value of the kullback leibler kl divergences between different classes based on this viewpoint the geometric mean for subspace selection is studied in this paper three criteria are analyzed 1 maximization of the geometric mean of the kl divergences 2 maximization of the geometric mean of the normalized kl divergences and 3 the combination of 1 and 2 preliminary experimental results based on synthetic data uci machine learning repository and handwriting digits show that the third criterion is a potential discriminative subspace selection method which significantly reduces the class separation problem in comparing with the linear dimensionality reduction step in flda and its several representative extensions

a variational method for learning sparse and overcomplete representations an expectation maximization algorithm for learning sparse and overcomplete data representations is presented the proposed algorithm exploits a variational approximation to a range of heavy tailed distributions whose limit is the laplacian a rigorous lower bound on the sparse prior distribution is derived which enables the analytic marginalization of a lower bound on the data likelihood this lower bound enables the development of an expectation maximization algorithm for learning the overcomplete basis vectors and inferring the most probable basis coefficients

discovering diverse subset for unsupervised hyperspectral band selection band selection as a special case of the feature selection problem tries to remove redundant bands and select a few important bands to represent the whole image cube this has attracted much attention since the selected bands provide discriminative information for further applications and reduce the computational burden though hyperspectral band selection has gained rapid development in recent years it is still a challenging task because of the following requirements 1 an effective model can capture the underlying relations between different high dimensional spectral bands 2 a fast and robust measure function can adapt to general hyperspectral tasks and 3 an efficient search strategy can find the desired selected bands in reasonable computational time to satisfy these requirements a multigraph determinantal point process mdpp model is proposed to capture the full structure between different bands and efficiently find the optimal band subset in extensive hyperspectral applications there are three main contributions 1 graphical model is naturally transferred to address band selection problem by the proposed mdpp 2 multiple graphs are designed to capture the intrinsic relationships between hyperspectral bands and 3 mixture dpp is proposed to model the multiple dependencies in the proposed multiple graphs and offers an efficient search strategy to select the optimal bands to verify the superiority of the proposed method experiments have been conducted on three hyperspectral applications such as hyperspectral classification anomaly detection and target detection the reliability of the proposed method in generic hyperspectral tasks is experimentally proved on four real world hyperspectral data sets

discriminative multimanifold analysis for face recognition from a single training sample per person conventional appearance based face recognition methods usually assume that there are multiple samples per person mspp available for discriminative feature extraction during the training phase in many practical face recognition applications such as law enhancement e passport and id card identification this assumption however may not hold as there is only a single sample per person sspp enrolled or recorded in these systems many popular face recognition methods fail to work well in this scenario because there are not enough samples for discriminant learning to address this problem we propose in this paper a novel discriminative multimanifold analysis dmma method by learning discriminative features from image patches first we partition each enrolled face image into several nonoverlapping patches to form an image set for each sample per person then we formulate the sspp face recognition as a manifold manifold matching problem and learn multiple dmma feature spaces to maximize the manifold margins of different persons finally we present a reconstruction based manifold manifold distance to identify the unlabeled subjects experimental results on three widely used face databases are presented to demonstrate the efficacy of the proposed approach

an unsupervised vector quantization based target subspace projection approach to mixed pixel detection and classification in unknown background for remotely sensed imagery a recently developed orthogonal subspace projection osp approach has been successfully applied to aviris as well as hydice data for image classification however it has found that osp performs poorly in multispectral image classification such as 3 band spot data this is primarily due to the fact that the number of signatures to be classified is greater than that of spectral bands used for data acquisition in which case the effects of uninterested signatures cannot be properly annihilated via orthogonal projection this constraint referred to as band number constraint bnc is generally not applied to hyperspectral images because the number of signatures resident within the images is usually far less than the total number of spectral bands in this paper a new approach called unsupervised vector quantization based target subspace projection uvqtsp is presented which can be implemented in an unknown environment with all required information obtained from the data to be processed the proposed uvqtsp has practical advantages over osp specifically it relaxes the band number constraint bnc so that it can be applied to multispectral imagery the uvqtsp uses vector quantization to find a set of clusters representing the unknown signatures and interferers which will be eliminated prior to target detection and classification the number of clusters can be determined by constraints such as the intrinsic dimensionality or the number of spectral bands this process is carried out in an unsupervised manner without training data the superiority of uvqtsp is demonstrated through real data including spot and hydice images

blind separation of mixture of independent sources through a quasi maximum likelihood approach we propose two methods for separating mixture of independent sources without any precise knowledge of their probability distribution they are obtained by considering a maximum likelihood ml solution corresponding to some given distributions of the sources and relaxing this assumption afterward the first method is specially adapted to temporally independent non gaussian sources and is based on the use of nonlinear separating functions the second method is specially adapted to correlated sources with distinct spectra and is based on the use of linear separating filters a theoretical analysis of the performance of the methods has been made a simple procedure for optimally choosing the separating functions is proposed further in the second method a simple implementation based on the simultaneous diagonalization of two symmetric matrices is provided finally some numerical and simulation results are given illustrating the performance of the method and the good agreement between the experiments and the theory

probabilistic distance for mixtures of independent component analyzers independent component analysis ica is a blind source separation technique where data are modeled as linear combinations of several independent non gaussian sources the independence and linear restrictions are relaxed using several ica mixture models icamms obtaining a two layer artificial neural network structure this allows for dependence between sources of different classes and thus a myriad of multidimensional probability density functions can be accurate modeled this paper proposes a new probabilistic distance pdi between the parameters learned for two icamms the pdi is computed explicitly unlike the popular kullback leibler divergence kld and other similar metrics removing the need for numerical integration furthermore the pdi is symmetric and bounded within 0 and 1 which enables its use as a posterior probability in fusion approaches in this paper the pdi is employed for change detection by measuring the distance between two icamms learned in consecutive time windows the changes might be associated with relevant states from a process under analysis that are explicitly reflected in the learned icamm parameters the proposed distance was tested in two challenging applications using simulated and real data 1 detecting flaws in materials using ultrasounds and 2 detecting changes in electroencephalography signals from humans performing neuropsychological tests the results demonstrate that the pdi outperforms the kld in change detection capabilities

robust locally linear embedding in the past few years some nonlinear dimensionality reduction nldr or nonlinear manifold learning methods have aroused a great deal of interest in the machine learning community these methods are promising in that they can automatically discover the low dimensional nonlinear manifold in a high dimensional data space and then embed the data points into a low dimensional embedding space using tractable linear algebraic techniques that are easy to implement and are not prone to local minima despite their appealing properties these nldr methods are not robust against outliers in the data yet so far very little has been done to address the robustness problem in this paper we address this problem in the context of an nldr method called locally linear embedding lle based on robust estimation techniques we propose an approach to make lle more robust we refer to this approach as robust locally linear embedding rlle we also present several specific methods for realizing this general rlle approach experimental results on both synthetic and real world data show that rlle is very robust against outliers

equivariant adaptive source separation source separation consists of recovering a set of independent signals when only mixtures with unknown coefficients are observed this paper introduces a class of adaptive algorithms for source separation that implements an adaptive version of equivariant estimation and is henceforth called equivariant adaptive separation via independence easi the easi algorithms are based on the idea of serial updating this specific form of matrix updates systematically yields algorithms with a simple structure for both real and complex mixtures most importantly the performance of an easi algorithm does not depend on the mixing matrix in particular convergence rates stability conditions and interference rejection levels depend only on the normalized distributions of the source signals closed form expressions of these quantities are given via an asymptotic performance analysis the theme of equivariance is stressed throughout the paper the source separation problem has an underlying multiplicative structure the parameter space forms a matrix multiplicative group we explore the favorable consequences of this fact on implementation performance and optimization of easi algorithms

discriminant nonnegative tensor factorization algorithms nonnegative matrix factorization nmf has proven to be very successful for image analysis especially for object representation and recognition nmf requires the object tensor with valence more than one to be vectorized this procedure may result in information loss since the local object structure is lost due to vectorization recently in order to remedy this disadvantage of nmf methods nonnegative tensor factorizations ntf algorithms that can be applied directly to the tensor representation of object collections have been introduced in this paper we propose a series of unsupervised and supervised ntf methods that is we extend several nmf methods using arbitrary valence tensors moreover by incorporating discriminant constraints inside the ntf decompositions we present a series of discriminant ntf methods the proposed approaches are tested for face verification and facial expression recognition where it is shown that they outperform other popular subspace approaches

a maximum likelihood approach to joint image registration and fusion both image registration and fusion can be formulated as estimation problems instead of estimating the registration parameters and the true scene separately as in the conventional way we propose a maximum likelihood approach for joint image registration and fusion in this paper more precisely the fusion performance is used as the criteria to evaluate the registration accuracy hence the registration parameters can be automatically tuned so that both fusion and registration can be optimized simultaneously the expectation maximization algorithm is employed to solve this joint optimization problem the cramer rao bound crb is then derived our experiments use several types of sensory images for performance evaluation such as visual images ir thermal images and hyperspectral images it is shown that the mean square error of estimating the registration parameters using the proposed method is close to the crbs at the mean time an improved fusion performance can be achieved in terms of the edge preservation measure qab f compared to the laplacian pyramid fusion approach

dimensionality reduction for hyperspectral data based on class aware tensor neighborhood graph and patch alignment to take full advantage of hyperspectral information to avoid data redundancy and to address the curse of dimensionality concern dimensionality reduction dr becomes particularly important to analyze hyperspectral data exploring the tensor characteristic of hyperspectral data a dr algorithm based on class aware tensor neighborhood graph and patch alignment is proposed here first hyperspectral data are represented in the tensor form through a window field to keep the spatial information of each pixel second using a tensor distance criterion a class aware tensor neighborhood graph containing discriminating information is obtained in the third step employing the patch alignment framework extended to the tensor space we can obtain global optimal spectral spatial information finally the solution of the tensor subspace is calculated using an iterative method and low dimensional projection matrixes for hyperspectral data are obtained accordingly the proposed method effectively explores the spectral and spatial information in hyperspectral data simultaneously experimental results on 3 real hyperspectral datasets show that compared with some popular vector and tensor based dr algorithms the proposed method can yield better performance with less tensor training samples required

variational learning of clusters of undercomplete nonsymmetric independent components we apply a variational method to automatically determine the number of mixtures of independent components in high dimensional datasets in which the sources may be nonsymmetrically distributed the data are modeled by clusters where each cluster is described as a linear mixture of independent factors the variational bayesian method yields an accurate density model for the observed data without overfitting problems this allows the dimensionality of the data to be identified for each cluster the new method was successfully applied to a difficult real world medical dataset for diagnosing glaucoma

selection and fusion of color models for image feature detection the choice of a color model is of great importance for many computer vision algorithms e g feature detection object recognition and tracking as the chosen color model induces the equivalence classes to the actual algorithms as there are many color models available the inherent difficulty is how to automatically select a single color model or alternatively a weighted subset of color models producing the best result for a particular task the subsequent hurdle is how to obtain a proper fusion scheme for the algorithms so that the results are combined in an optimal setting to achieve proper color model selection and fusion of feature detection algorithms in this paper we propose a method that exploits nonperfect correlation between color models or feature detection algorithms derived from the principles of diversification as a consequence a proper balance is obtained between repeatability and distinctiveness the result is a weighting scheme which yields maximal feature discrimination the method is verified experimentally for three different image feature detectors the experimental results show that the fusion method provides feature detection results having a higher discriminative power than the standard weighting scheme further it is experimentally shown that the color model selection scheme provides a proper balance between color invariance repeatability and discriminative power distinctiveness

variational mixture of bayesian independent component analyzers there has been growing interest in subspace data modeling over the past few years methods such as principal component analysis factor analysis and independent component analysis have gained in popularity and have found many applications in image modeling signal processing and data compression to name just a few as applications and computing power grow more and more sophisticated analyses and meaningful representations are sought mixture modeling methods have been proposed for principal and factor analyzers that exploit local gaussian features in the subspace manifolds meaningful representations may be lost however if these local features are nongaussian or discontinuous in this article we propose extending the gaussian analyzers mixture model to an independent component analyzers mixture model we employ recent developments in variational bayesian inference and structure determination to construct a novel approach for modeling nongaussian discontinuous manifolds we automatically determine the local dimensionality of each manifold and use variational inference to calculate the optimum number of ica components needed in our mixture model we demonstrate our framework on complex synthetic data and illustrate its application to real data by decomposing functional magnetic resonance images into meaningful and medically useful features

orthogonal least squares learning algorithm for radial basis function networks the radial basis function network offers a viable alternative to the two layer neural network in many applications of signal processing a common learning algorithm for radial basis function networks is based on first choosing randomly some data points as radial basis function centers and then using singular value decomposition to solve for the weights of the network such a procedure has several drawbacks and in particular an arbitrary selection of centers is clearly unsatisfactory the authors propose an alternative learning procedure based on the orthogonal least squares method the procedure chooses radial basis function centers one by one in a rational way until an adequate network has been constructed in the algorithm each selected center maximizes the increment to the explained variance or energy of the desired output and does not suffer numerical ill conditioning problems the orthogonal least squares learning strategy provides a simple and efficient means for fitting radial basis function networks this is illustrated using examples taken from two different signal processing applications

multilinear discriminant analysis for face recognition there is a growing interest in subspace learning techniques for face recognition however the excessive dimension of the data space often brings the algorithms into the curse of dimensionality dilemma in this paper we present a novel approach to solve the supervised dimensionality reduction problem by encoding an image object as a general tensor of second or even higher order first we propose a discriminant tensor criterion whereby multiple interrelated lower dimensional discriminative subspaces are derived for feature extraction then a novel approach called k mode optimization is presented to iteratively learn these subspaces by unfolding the tensor along different tensor directions we call this algorithm multilinear discriminant analysis mda which has the following characteristics 1 multiple interrelated subspaces can collaborate to discriminate different classes 2 for classification problems involving higher order tensors the mda algorithm can avoid the curse of dimensionality dilemma and alleviate the small sample size problem and 3 the computational cost in the learning stage is reduced to a large extent owing to the reduced data dimensions in k mode optimization we provide extensive experiments on orl cmu pie and feret databases by encoding face images as second or third order tensors to demonstrate that the proposed mda algorithm based on higher order tensors has the potential to outperform the traditional vector based subspace learning algorithms especially in the cases with small sample sizes

independent factor analysis we introduce the independent factor analysis ifa method for recovering independent hidden sources from their observed mixtures ifa generalizes and unifies ordinary factor analysis fa principal component analysis pca and independent component analysis ica and can handle not only square noiseless mixing but also the general case where the number of mixtures differs from the number of sources and the data are noisy ifa is a two step procedure in the first step the source densities mixing matrix and noise covariance are estimated from the observed data by maximum likelihood for this purpose we present an expectation maximization em algorithm which performs unsupervised learning of an associated probabilistic model of the mixing situation each source in our model is described by a mixture of gaussians thus all the probabilistic calculations can be performed analytically in the second step the sources are reconstructed from the observed data by an optimal nonlinear estimator a variational approximation of this algorithm is derived for cases with a large number of sources where the exact algorithm becomes intractable our ifa algorithm reduces to the one for ordinary fa when the sources become gaussian and to an em algorithm for pca in the zero noise limit we derive an additional em algorithm specifically for noiseless ifa this algorithm is shown to be superior to ica since it can learn arbitrary source densities from the data beyond blind separation ifa can be used for modeling multidimensional data by a highly constrained mixture of gaussians and as a tool for nonlinear signal encoding

mpca multilinear principal component analysis of tensor objects this paper introduces a multilinear principal component analysis mpca framework for tensor object feature extraction objects of interest in many computer vision and pattern recognition applications such as 2d 3d images and video sequences are naturally described as tensors or multilinear arrays the proposed framework performs feature extraction by determining a multilinear projection that captures most of the original tensorial input variation the solution is iterative in nature and it proceeds by decomposing the original problem to a series of multiple projection subproblems as part of this work methods for subspace dimensionality determination are proposed and analyzed it is shown that the mpca framework discussed in this work supplants existing heterogeneous solutions such as the classical principal component analysis pca and its 2d variant 2d pca finally a tensor object recognition system is proposed with the introduction of a discriminative tensor feature selection mechanism and a novel classification strategy and applied to the problem of gait recognition results presented here indicate mpca s utility as a feature extraction tool it is shown that even without a fully optimized design an mpca based gait recognition module achieves highly competitive performance and compares favorably to the state of the art gait recognizers

mr ntd manifold regularization nonnegative tucker decomposition for tensor data dimension reduction and representation with the advancement of data acquisition techniques tensor multidimensional data objects are increasingly accumulated and generated for example multichannel electroencephalographies multiview images and videos in these applications the tensor objects are usually nonnegative since the physical signals are recorded as the dimensionality of tensor objects is often very high a dimension reduction technique becomes an important research topic of tensor data from the perspective of geometry high dimensional objects often reside in a low dimensional submanifold of the ambient space in this paper we propose a new approach to perform the dimension reduction for nonnegative tensor objects our idea is to use nonnegative tucker decomposition ntd to obtain a set of core tensors of smaller sizes by finding a common set of projection matrices for tensor objects to preserve geometric information in tensor data we employ a manifold regularization term for the core tensors constructed in the tucker decomposition an algorithm called manifold regularization ntd mr ntd is developed to solve the common projection matrices and core tensors in an alternating least squares manner the convergence of the proposed algorithm is shown and the computational complexity of the proposed method scales linearly with respect to the number of tensor objects and the size of the tensor objects respectively these theoretical results show that the proposed algorithm can be efficient extensive experimental results have been provided to further demonstrate the effectiveness and efficiency of the proposed mr ntd algorithm

face recognition using an enhanced independent component analysis approach this paper is concerned with an enhanced independent component analysis ica and its application to face recognition typically face representations obtained by ica involve unsupervised learning and high order statistics in this paper we develop an enhancement of the generic ica by augmenting this method by the fisher linear discriminant analysis lda hence its abbreviation fica the fica is systematically developed and presented along with its underlying architecture a comparative analysis explores four distance metrics as well as classification with support vector machines svms we demonstrate that the fica approach leads to the formation of well separated classes in low dimension subspace and is endowed with a great deal of insensitivity to large variation in illumination and facial expression the comprehensive experiments are completed for the facial recognition technology feret face database a comparative analysis demonstrates that fica comes with improved classification rates when compared with some other conventional approaches such as eigenface fisherface and the ica itself

mode kn factor analysis for image ensembles in this corespondence we study the extra factor estimation problem with the assumption that the training image ensemble is expressed as an nth order tensor with the nth dimension characterizing all features for an image and other dimensions for different extra factors such as illuminations poses and identities to overcome the local minimum issue of conventional algorithms designed for this problem we present a novel statistical learning framework called mode kn factor analysis for obtaining a closed form solution to estimating the extra factors of any test image in the learning stage for the kth k ne n dimension of the data tensor the mode kn patterns are constructed by concatenating the feature dimension and the kth extra factor dimension and then a mode kn factor analysis model is learnt based on the mode kn patterns unfolded from the original data tensor in the inference stage for a test image the mode classification of the kth dimension is performed within a probabilistic framework the advantages of mode kn factor analysis over conventional tensor analysis algorithms are twofold 1 a closed form solution instead of iterative sub optimal solution as conventionally is derived for estimating the extra factor mode of any test image and 2 the classification capability is enhanced by interacting with the process of synthesizing data of all other modes in the k th dimension experiments on the pointing 04 and cmu pie databases for pose and illumination estimation both validate the superiority of the proposed algorithm over conventional algorithms for extra factor estimation

an information maximization approach to blind separation and blind deconvolution we derive a new self organizing learning algorithm that maximizes the information transferred in a network of nonlinear units the algorithm does not assume any knowledge of the input distributions and is defined here for the zero noise limit under these conditions information maximization has extra properties not found in the linear case linsker 1989 the nonlinearities in the transfer function are able to pick up higher order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation this enables the network to separate statistically independent components in the inputs a higher order generalization of principal components analysis we apply the network to the source separation or cocktail party problem successfully separating unknown mixtures of up to 10 speakers we also show that a variant on the network architecture is able to perform blind deconvolution cancellation of unknown echoes and reverberation in a speech signal finally we derive dependencies of information transfer on time delays we suggest that information maximization provides a unifying framework for problems in blind signal processing

the random subspace method for constructing decision forests much of previous attention on decision trees focuses on the splitting criteria and optimization of tree sizes the dilemma between overfitting and achieving maximum accuracy is seldom resolved a method to construct a decision tree based classifier is proposed that maintains highest accuracy on training data and improves on generalization accuracy as it grows in complexity the classifier consists of multiple trees constructed systematically by pseudorandomly selecting subsets of components of the feature vector that is trees constructed in randomly chosen subspaces the subspace method is compared to single tree classifiers and other forest construction methods by experiments on publicly available datasets where the method s superiority is demonstrated we also discuss independence between trees in a forest and relate that to the combined classification accuracy

tensor learning for regression in this paper we exploit the advantages of tensorial representations and propose several tensor learning models for regression the model is based on the canonical parallel factor decomposition of tensors of multiple modes and allows the simultaneous projections of an input tensor to more than one direction along each mode two empirical risk functions are studied namely the square loss and e insensitive loss functions the former leads to higher rank tensor ridge regression trr and the latter leads to higher rank support tensor regression str both formulated using the frobenius norm for regularization we also use the group sparsity norm for regularization favoring in that way the low rank decomposition of the tensorial weight in that way we achieve the automatic selection of the rank during the learning process and obtain the optimal rank trr and str experiments conducted for the problems of head pose human age and 3 d body pose estimations using real data from publicly available databases verified not only the superiority of tensors over their vector counterparts but also the efficiency of the proposed algorithms

some nonlinear maps and renormings of banach spaces we consider two classes of nonlinear maps between normed spaces which are relevant for the study of banach spaces that admit equivalent locally uniformly rotund and midpoint locally uniformly rotund norms it turns out to be a useful characterization of such maps in terms of optimization topology and probability using these characterizations we obtain some renorming results

blind separation of independent sources using gaussian mixture model in this paper two novel algorithms for blind separation of noiseless instantaneous linear mixture of independent sources are presented the proposed algorithms exploit nongaussianity of the independent sources by modeling their distribution using the gaussian mixture model gmm the first proposed method is based on the maximum likelihood ml estimator according to this method the sensors distribution parameters are estimated via the expectation maximization em algorithm for gmm parameter estimation and the separation matrix is estimated by applying nonorthogonal joint diagonalization of the estimated gmm covariance matrices the second proposed method is also a ml based approach according to this method the distribution parameters of the prewhitened sensors are estimated via the em algorithm for gmm parameter estimation and a unitary separation matrix is estimated by applying orthogonal joint diagonalization of the estimated gmm covariance matrices it is shown that estimation of the sensors distribution parameters amounts to obtaining a tight lower bound on the log likelihood of the separation matrix and that the joint diagonalization of the estimated gmm covariance matrices amounts to maximization of the obtained tight lower bound simulations demonstrate that the proposed methods outperform state of the art blind source separation techniques in terms of interference to signal ratio