fast calculation of laurent expansions for matrix inverses previously described algorithms for calculating the laurent expansion of the inverse of a matrix valued analytic function become impractical already for singularity orders as low as around p 6 since they require over o 28p matrix multiplications and correspondingly large amounts of memory in place of using mathematically exact recursions we show that for floating point calculations a rational approximation approach can avoid this cost barrier without any significant loss in accuracy

stable computation of differentiation matrices and scattered node stencils based on gaussian radial basis functions stable computation of differentiation matrices and scattered node stencils based on gaussian radial basis functions

tensor representation learning based image patch analysis for text identification and recognition in this paper we introduce a novel framework for text identification and recognition called tensor representation learning based image patch analysis trl ipa unlike most of previous text identification approaches which can only be applied to binarized images trl ipa can be directly applied to gray level and color images trl ipa is built on a general formulation of the convergent tensor representation learning ctrl algorithms in the implementation of trl ipa image patches are represented in the form of tensors while low dimensional representations of these tensors are learned via a ctrl algorithm to identify text regions in new coming document images a random forest classifier is trained in the learned tensor subspace moreover the trl ipa framework can be straightforwardly applied to recognition problems such as handwritten digits recognition we conducted extensive experiments on ancient chinese arabic and cyrillic document images to evaluate trl ipa on text identification tasks experimental results demonstrate its effectiveness and robustness in addition recognition results on images of handwritten digits show its advantage over state of the art vector and tensor representation based approaches highlightsa novel model tensor representation learning based image patch analysis trl ipa is proposed for document understanding trl ipa is built on a general formulation of the convergent tensor representation learning ctrl algorithms the ctrl algorithms are theoretically guaranteed to converge to a local optimal solution of the learning problem extensive experiments demonstrate the superiority of trl ipa over related vector and tensor representation based approaches

analytic perturbation of generalized inverses abstract we investigate the analytic perturbation of generalized inverses firstly we analyze the analytic perturbation of the drazin generalized inverse also known as reduced resolvent in operator theory our approach is based on spectral theory of linear operators as well as on a new notion of group reduced resolvent it allows us to treat regular and singular perturbations in a unified framework we provide an algorithm for computing the coefficients of the laurent series of the perturbed drazin generalized inverse in particular the regular part coefficients can be efficiently calculated by recursive formulae finally we apply the obtained results to the perturbation analysis of the moore penrose generalized inverse in the real domain

complete discrete 2 d gabor transforms by neural networks for image analysis and compression a three layered neural network is described for transforming two dimensional discrete signals into generalized nonorthogonal 2 d gabor representations for image analysis segmentation and compression these transforms are conjoint spatial spectral representations which provide a complete image description in terms of locally windowed 2 d spectral coordinates embedded within global 2 d spatial coordinates in the present neural network approach based on interlaminar interactions involving two layers with fixed weights and one layer with adjustable weights the network finds coefficients for complete conjoint 2 d gabor transforms without restrictive conditions in wavelet expansions based on a biologically inspired log polar ensemble of dilations rotations and translations of a single underlying 2 d gabor wavelet template image compression is illustrated with ratios up to 20 1 also demonstrated is image segmentation based on the clustering of coefficients in the complete 2 d gabor transform

theoretical and computational aspects of multivariate interpolation with increasingly flat radial basis functions multivariate interpolation of smooth data using smooth radial basis functions is considered the behavior of the interpolants in the limit of nearly flat radial basis functions is studied both theoretically and numerically explicit criteria for different types of limits are given using the results for the limits the dependence of the error on the shape parameter of the radial basis function is investigated the mechanisms that determine the optimal shape parameter value are studied and explained through approximate expansions of the interpolation error

a numerical study of some radial basis function based solution methods for elliptic pdes abstract during the last decade three main variations have been proposed for solving elliptic pdes by means of collocation with radial basis functions rbfs in this study we have implemented them for infinitely smooth rbfs and then compared them across the full range of values for the shape parameter of the rbfs this was made possible by a recently discovered numerical procedure that bypasses the ill conditioning which has previously limited the range that could be used for this parameter we find that the best values for it often fall outside the range that was previously available we have also looked at piecewise smooth versus infinitely smooth rbfs and found that for pde applications with smooth solutions the infinitely smooth rbfs are preferable mainly because they lead to higher accuracy in a comparison of rbf based methods against two standard techniques a second order finite difference method and a pseudospectral method the former gave a much superior accuracy

stable computation of multiquadric interpolants for all values of the shape parameter abstract spectrally accurate interpolation and approximation of derivatives used to be practical only on highly regular grids in very simple geometries since radial basis function rbf approxima ions permit this even for multivariate scattered data there has been much recent interest in practical algorithms to compute these approximations effectively several types of rbfs feature a free parameter e g c in the multiquadric mq case f r r 2 c 2 the limit of c increasingly flat basis functions has not received much attention because it leads to a severely ill conditioned problem we present here an algorithm which avoids this difficulty and which allows numerically stable computations of mq rbf interpolants for all parameter values we then find that the accuracy of the resulting approximations in some cases becomes orders of magnitude higher than was the case within the previously available parameter range our new method provides the first tool for the numerical exploration of mq rbf interpolants in the limit of c the method is in no way specific to mq basis functions and can without any change be applied to many other cases as well

general tensor discriminant analysis and gabor features for gait recognition traditional image representations are not suited to conventional classification methods such as the linear discriminant analysis lda because of the undersample problem usp the dimensionality of the feature space is much higher than the number of training samples motivated by the successes of the two dimensional lda 2dlda for face recognition we develop a general tensor discriminant analysis gtda as a preprocessing step for lda the benefits of gtda compared with existing preprocessing methods such as the principal components analysis pca and 2dlda include the following 1 the usp is reduced in subsequent classification by for example lda 2 the discriminative information in the training tensors is preserved and 3 gtda provides stable recognition rates because the alternating projection optimization algorithm to obtain a solution of gtda converges whereas that of 2dlda does not we use human gait recognition to validate the proposed gtda the averaged gait images are utilized for gait representation given the popularity of gabor function based image decompositions for image understanding and object recognition we develop three different gabor function based image representations 1 gabord is the sum of gabor filter responses over directions 2 gabors is the sum of gabor filter responses over scales and 3 gaborsd is the sum of gabor filter responses over scales and directions the gabord gabors and gaborsd representations are applied to the problem of recognizing people from their averaged gait images a large number of experiments were carried out to evaluate the effectiveness recognition rate of gait recognition based on first obtaining a gabor gabord gabors or gaborsd image representation then using gdta to extract features and finally using lda for classification the proposed methods achieved good performance for gait recognition based on image sequences from the university of south florida usf humanid database experimental comparisons are made with nine state of the art classification methods in gait recognition

randomized algorithms for robust controller synthesis using statistical learning theory by now it is known that several problems in the robustness analysis and synthesis of control systems are np complete or np hard these negative results force us to modify our notion of solving a given problem an approach that is recently gaining popularity is that of using randomized algorithms which can be used to solve a problem approximately most of the time we begin with the premise that many problems in robustness analysis and synthesis can be formulated as the minimization of an objective function with respect to the controller parameters it is argued that in order to assess the performance of a controller as the plant varies over a prespecified family it is better to use the average performance of the controller as the objective function to be minimized rather than its worst case performance as the worst case objective function usually leads to rather conservative designs then it is shown that a property from statistical learning theory known as uniform convergence of empirical means ucem plays an important role in allowing us to construct efficient randomized algorithms for a wide variety of controller synthesis problems in particular whenever the ucem property holds there exists an efficient i e polynomial time randomized algorithm using very recent results in statistical learning theory it is shown that the ucem property holds in any problem in which the satisfaction of a performance constraint can be expressed in terms of a finite number of polynomial inequalities in particular several problems such as robust stabilization and weighted h 2 h norm minimization are amenable to the randomized approach

large margin low rank tensor analysis we present a supervised model for tensor dimensionality reduction which is called large margin low rank tensor analysis lmlrta in contrast to traditional vector representation based dimensionality reduction methods lmlrta can take any order of tensors as input and unlike previous tensor dimensionality reduction methods which can learn only the low dimensional embeddings with a priori specified dimensionality lmlrta can automatically and jointly learn the dimensionality and the low dimensional representations from data moreover lmlrta delivers low rank projection matrices while it encourages data of the same class to be close and of different classes to be separated by a large margin of distance in the low dimensional tensor space lmlrta can be optimized using an iterative fixed point continuation algorithm which is guaranteed to converge to a local optimal solution of the optimization problem we evaluate lmlrta on an object recognition application where the data are represented as 2d tensors and a face recognition application where the data are represented as 3d tensors experimental results show the superiority of lmlrta over state of the art approaches

the runge phenomenon and spatially variable shape parameters in rbf interpolation many studies mostly empirical have been devoted to finding an optimal shape parameter for radial basis functions rbf when exploring the underlying factors that determine what is a good such choice we are led to consider the runge phenomenon rp best known in cases of high order polynomial interpolation as a key error mechanism this observation suggests that it can be advantageous to let the shape parameter vary spatially rather than assigning a single value to it benefits typically include improvements in both accuracy and numerical conditioning still another benefit arises if one wishes to improve local accuracy by clustering nodes in selected areas this idea is routinely used when working with splines or finite element methods however local refinement with rbfs may cause rp type errors unless we use a spatially variable shape paremeter with this enhancement rbf approximations combine freedom from meshes with spectral accuracy on irregular domains and furthermore permit local node clustering to improve the resolution wherever this might be needed

image classification using correlation tensor analysis images as high dimensional data usually embody large variabilities to classify images for versatile applications an effective algorithm is necessarily designed by systematically considering the data structure similarity metric discriminant subspace and classifier in this paper we provide evidence that besides the fisher criterion graph embedding and tensorization used in many existing methods the correlation based similarity metric embodied in supervised multilinear discriminant subspace learning can additionally improve the classification performance in particular a novel discriminant subspace learning algorithm called correlation tensor analysis cta is designed to incorporate both graph embedded correlational mapping and discriminant analysis in a fisher type of learning manner the correlation metric can estimate intrinsic angles and distances for the locally isometric embedding which can deal with the case when euclidean metric is incapable of capturing the intrinsic similarities between data points cta learns multiple interrelated subspaces to obtain a low dimensional data representation reflecting both class label information and intrinsic geometric structure of the data distribution extensive comparisons with most popular subspace learning methods on face recognition evaluation demonstrate the effectiveness and superiority of cta parameter analysis also reveals its robustness

observations on the behavior of radial basis function approximations near boundaries abstract rbf approximations would appear to be very attractive for approximating spatial derivatives in numerical simulations of pdes rbfs allow arbitrarily scattered data generalize easily to several space dimensions and can be spectrally accurate however accuracy degradations near boundaries in many cases severely limit the utility of this approach with that as motivation this study aims at gaining a better understanding of the properties of rbf approximations near the ends of an interval in 1 d and towards edges in 2 d

the exponentially convergent trapezoidal rule it is well known that the trapezoidal rule converges geometrically when applied to analytic functions on periodic intervals or the real line the mathematics and history of this phenomenon are reviewed and it is shown that far from being a curiosity it is linked with computational methods all across scientific computing including algorithms related to inverse laplace transforms special functions complex analysis rational approximation integral equations and the computation of functions and eigenvalues of matrices and operators

stable computations with flat radial basis functions using vector valued rational approximations one commonly finds in applications of smooth radial basis functions rbfs that scaling the kernels so they are flat leads to smaller discretization errors however the direct numerical approach for computing with flat rbfs rbf direct is severely ill conditioned we present an algorithm for bypassing this ill conditioning that is based on a new method for rational approximation ra of vector valued analytic functions with the property that all components of the vector share the same singularities this new algorithm rbf ra is more accurate robust and easier to implement than the contour pade method which is similarly based on vector valued rational approximation in contrast to the stable rbf qr and rbf ga algorithms which are based on finding a better conditioned base in the same rbf space the new algorithm can be used with any type of smooth radial kernel and it is also applicable to a wider range of tasks including calculating hermite type implicit rbf fd stencils we present a series of numerical experiments demonstrating the effectiveness of this new method for computing rbf interpolants in the flat regime we also demonstrate the flexibility of the method by using it to compute implicit rbf fd formulas in the flat regime and then using these for solving poisson s equation in a 3 d spherical shell

probabilistic non linear principal component analysis with gaussian process latent variable models summarising a high dimensional data set with a low dimensional embedding is a standard approach for exploring its structure in this paper we provide an overview of some existing techniques for discovering such embeddings we then introduce a novel probabilistic interpretation of principal component analysis pca that we term dual probabilistic pca dppca the dppca model has the additional advantage that the linear mappings from the embedded space can easily be non linearised through gaussian processes we refer to this model as a gaussian process latent variable model gp lvm through analysis of the gp lvm objective function we relate the model to popular spectral techniques such as kernel pca and multidimensional scaling we then review a practical algorithm for gp lvms in the context of large data sets and develop it to also handle discrete valued data and missing attributes we demonstrate the model on a range of real world and artificially generated data sets

graph embedding and extensions a general framework for dimensionality reduction a large family of algorithms supervised or unsupervised stemming from statistics or geometry theory has been designed to provide different solutions to the problem of dimensionality reduction despite the different motivations of these algorithms we present in this paper a general formulation known as graph embedding to unify them within a common framework in graph embedding each algorithm can be considered as the direct graph embedding or its linear kernel tensor extension of a specific intrinsic graph that describes certain desired statistical or geometric properties of a data set with constraints from scale normalization or a penalty graph that characterizes a statistical or geometric property that should be avoided furthermore the graph embedding framework can be used as a general platform for developing new dimensionality reduction algorithms by utilizing this framework as a tool we propose a new supervised dimensionality reduction algorithm called marginal fisher analysis in which the intrinsic graph characterizes the intraclass compactness and connects each data point with its neighboring points of the same class while the penalty graph connects the marginal points and characterizes the interclass separability we show that mfa effectively overcomes the limitations of the traditional linear discriminant analysis algorithm due to data distribution assumptions and available projection directions real face recognition experiments show the superiority of our proposed mfa in comparison to lda also for corresponding kernel and tensor extensions

froissart doublets in pade approximation in the case of polynomial noise first we study the relation between the zeros of random polynomials rn 1 and the zeros and poles of their pade approximants n n rn 1 next we consider the distribution of zeros and poles of pade approximants to the geometric series perturbed by a random polynomial noise we observe numerically interesting connections between two above problems some numerical observations on the froissart doublets have been also made

robust h control design for best mean performance over an uncertain parameters box abstract this paper deals with robust polytopic probabilistic h analysis and state feedback synthesis of linear systems and focuses on the performance distribution over the uncertainty region rather than on the performance bound the proposed approach allows different disturbance attenuation levels dals at the vertices of the uncertainty polytope it is shown that the mean disturbance attenuation level dal over an uncertain parameters box is the average of the dals at the vertices if each parameter has an independent symmetrical centered probability density function in such a most common case the mean dal over the uncertain parameters box can be optimized by minimizing the sum of the dals at the vertices the standard deviation of the dal over the uncertain parameters box is also addressed and a method to minimize this standard deviation is shown a new robust h state feedback synthesis theorem is given it is based on a recent most efficient analysis method and applies the proposed multiple vertex dals approach a state feedback design example utilizing the latter theorem shows that a control design which minimizes the sum of the vertex dals leads to a better actual closed loop performance than a similar design which minimizes only the bound of the dal over the uncertainty polytope the comparison is based on the statistics of a population of closed loop point wise h norms created by a monte carlo mechanism

nonlinear component analysis as a kernel eigenvalue problem a new method for performing a nonlinear form of principal component analysis is proposed by the use of integral operator kernel functions one can efficiently compute principal components in high dimensional feature spaces related to input space by some nonlinear map for instance the space of all possible five pixel products in 16 16 images we give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition

probability guaranteed h finite horizon filtering for a class of nonlinear time varying systems with sensor saturations in this paper the probability guaranteed h finite horizon filtering problem is investigated for a class of nonlinear time varying systems with uncertain parameters and sensor saturations the system matrices are functions of mutually independent stochastic variables that obey uniform distributions over known finite ranges attention is focused on the construction of a time varying filter such that the prescribed h performance requirement can be guaranteed with probability constraint by using the difference linear matrix inequalities dlmis approach sufficient conditions are established to guarantee the desired performance of the designed finite horizon filter the time varying filter gains can be obtained in terms of the feasible solutions of a set of dlmis that can be recursively solved by using the semi definite programming method a computational algorithm is specifically developed for the addressed probability guaranteed h finite horizon filtering problem finally a simulation example is given to illustrate the effectiveness of the proposed filtering scheme 2012 elsevier b v all rights reserved

tensor decompositions and applications this survey provides an overview of higher order tensor decompositions their applications and available software a tensor is a multidimensional or n way array decompositions of higher order tensors i e n way arrays with n geq 3 have applications in psycho metrics chemometrics signal processing numerical linear algebra computer vision numerical analysis data mining neuroscience graph analysis and elsewhere two particular tensor decompositions can be considered to be higher order extensions of the matrix singular value decomposition candecomp parafac cp decomposes a tensor as a sum of rank one tensors and the tucker decomposition is a higher order form of principal component analysis there are many other tensor decompositions including indscal parafac2 candelinc dedicom and paratuck2 as well as nonnegative variants of all of the above the n way toolbox tensor toolbox and multilinear engine are examples of software packages for working with tensors

low rank tensor subspace learning for rgb d action recognition rgb d action data inherently equip with extra depth information to improve performance of action recognition compared with rgb data and many works represent the rgb d data as a third order tensor containing a spatiotemporal structure and find a subspace with lower dimension however there are two main challenges of these methods first the dimension of subspace is usually fixed manually which may not describe the samples well in the subspace second preserving local information by finding the intra class and inter class neighbors from a manifold is highly time consuming in this paper we learn a tensor subspace whose dimension is learned automatically by low rank learning for rgb d action recognition particularly the tensor samples are factorized to obtain three projection matrices pms by tucker decomposition where all the pms are performed by nuclear norm in a close form to obtain the tensor ranks which are used as tensor subspace dimension in addition we extract the discriminant and local information from a manifold using a graph constraint this graph preserves the local knowledge inherently which is faster than the previous way of calculating both the intra class and inter class neighbors of each sample we evaluate the proposed method on four widely used rgb d action datasets including msrdailyactivity3d msractionpairs msractionpairs skeleton and utkinect action3d datasets and the experimental results show higher accuracy and efficiency of the proposed method

a fast learning algorithm for deep belief nets we show how to use complementary priors to eliminate the explaining away effects that make inference difficult in densely connected belief nets that have many hidden layers using complementary priors we derive a fast greedy algorithm that can learn deep directed belief networks one layer at a time provided the top two layers form an undirected associative memory the fast greedy algorithm is used to initialize a slower learning procedure that fine tunes the weights using a contrastive version of the wake sleep algorithm after fine tuning a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels this generative model gives better digit classification than the best discriminative learning algorithms the low dimensional manifolds on which the digits lie are modeled by long ravines in the free energy landscape of the top level associative memory and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind

stabilization of uncertain systems with norm bounded uncertainty a control a robust stabilization problem in a state space setting is treated it is assumed that the states are available for feedback using a fixed lyapunov function approach quadratic stability it is shown that an open loop stabilizability condition is equivalent to the existence of a stabilizing memoryless linear state feedback controller as a consequence it is shown that the existence of a quadratically stabilizing nonlinear time varying dynamic state feedback controller implies the existence of a quadratically stabilizing memoryless linear time invariant state feedback compensator

a stable algorithm for flat radial basis functions on a sphere when radial basis functions rbfs are made increasingly flat the interpolation error typically decreases steadily until some point when runge type oscillations either halt or reverse this trend because the most obvious method to calculate an rbf interpolant becomes a numerically unstable algorithm for a stable problem in the case of near flat basis functions there will typically also be a separate point at which disastrous ill conditioning enters we introduce here a new method rbf qr which entirely eliminates such ill conditioning and we apply it in the special case when the data points are distributed over the surface of a sphere this algorithm works even for thousands of node points and it allows the rbf shape parameter to be optimized without the limitations imposed by stability concerns since interpolation in the flat rbf limit on a sphere is found to coincide with spherical harmonics interpolation new insights are gained as to why the rbf approach with nonflat basis functions often is the more accurate of the two methods

frequency optimized rbf fd for wave equations abstract we present a method to obtain optimal rbf fd formulas which maximize their frequency range of validity the optimization is based on the idea of keeping an error of interest dispersion phase or group velocity errors below a given threshold for a wavenumber interval as large as possible to find the weights of these optimal finite difference formulas we solve an optimization problem in a previous work we developed a method to optimize the frequency range of validity for finite difference weights that method required to solve a system of nonlinear equations with as many unknowns as half of the number of weights which is a very hard task when the number of nodes gets large the current method requires solving an optimization problem with only one parameter which makes finding a global minimum easier and thus can be used for bigger stencils we also study which of the standard rbf are more appropriate for this problem and introduce a new rbf that depends on two parameters this new rbf improves the resulting frequency response of the rbf fd methods while keeping the cost of the optimization problem low

interpolation in the limit of increasingly flat radial basis functions abstract many types of radial basis functions such as multiquadrics contain a free parameter in the limit where the basis functions become increasingly flat the linear system to solve becomes highly ill conditioned and the expansion coefficients diverge nevertheless we find in this study that limiting interpolants often exist and take the form of polynomials in the 1 d case we prove that with simple conditions on the basis function the interpolants converge to the lagrange interpolating polynomial hence differentiation of this limit is equivalent to the standard finite difference method we also summarize some preliminary observations regarding the limit in 2 d

stable computations with gaussian radial basis functions radial basis function rbf approximation is an extremely powerful tool for representing smooth functions in nontrivial geometries since the method is mesh free and can be spectrally accurate a perceived practical obstacle is that the interpolation matrix becomes increasingly ill conditioned as the rbf shape parameter becomes small corresponding to flat rbfs two stable approaches that overcome this problem exist the contour pade method and the rbf qr method however the former is limited to small node sets and the latter has until now been formulated only for the surface of the sphere this paper focuses on an rbf qr formulation for node sets in one two and three dimensions the algorithm is stable for arbitrarily small shape parameters it can be used for thousands of node points in two dimensions and still more in three dimensions a sample matlab code for the two dimensional case is provided

laurent expansion of the inverse of perturbed singular matrices in this paper we describe a numerical algorithm to compute the laurent expansion of the inverse of singularly perturbed matrices the algorithm is based on the resolvent formalism used in complex analysis to study the spectrum of matrices the input of the algorithm are the matrix coefficients of the power series expansion of the perturbed matrix the matrix coefficients of the laurent expansion of the inverse are computed using recursive analytical formulae we show that the computational complexity of the proposed algorithm grows algebraically with the size of the matrix but exponentially with the order of the singularity we apply this algorithm to several matrices that arise in applications we make special emphasis to interpolation problems with radial basis functions

dimensionality reduction of multimodal labeled data by local fisher discriminant analysis reducing the dimensionality of data without losing intrinsic information is an important preprocessing step in high dimensional data analysis fisher discriminant analysis fda is a traditional technique for supervised dimensionality reduction but it tends to give undesired results if samples in a class are multimodal an unsupervised dimensionality reduction method called locality preserving projection lpp can work well with multimodal data due to its locality preserving property however since lpp does not take the label information into account it is not necessarily useful in supervised learning scenarios in this paper we propose a new linear supervised dimensionality reduction method called local fisher discriminant analysis lfda which effectively combines the ideas of fda and lpp lfda has an analytic form of the embedding transformation and the solution can be easily computed just by solving a generalized eigenvalue problem we demonstrate the practical usefulness and high scalability of the lfda method in data visualization and classification tasks through extensive simulation studies we also show that lfda can be extended to non linear dimensionality reduction scenarios by applying the kernel trick

radial basis function interpolation in the limit of increasingly flat basis functions abstract we propose a new approach to study radial basis function rbf interpolation in the limit of increasingly flat functions the new approach is based on the semi analytical computation of the laurent series of the inverse of the rbf interpolation matrix described in a previous paper 3 once the laurent series is obtained it can be used to compute the limiting polynomial interpolant the optimal shape parameter of the rbfs used for interpolation and the weights of rbf finite difference formulas among other things

stacked denoising autoencoders learning useful representations in a deep network with a local denoising criterion we explore an original strategy for building deep networks based on stacking layers of denoising autoencoders which are trained locally to denoise corrupted versions of their inputs the resulting algorithm is a straightforward variation on the stacking of ordinary autoencoders it is however shown on a benchmark of classification problems to yield significantly lower classification error thus bridging the performance gap with deep belief networks dbn and in several cases surpassing it higher level representations learnt in this purely unsupervised fashion also help boost the performance of subsequent svm classifiers qualitative experiments show that contrary to ordinary autoencoders denoising autoencoders are able to learn gabor like edge detectors from natural image patches and larger stroke detectors from digit images this work clearly establishes the value of using a denoising criterion as a tractable unsupervised objective to guide the learning of useful higher level representations

bayesian cp factorization of incomplete tensors with automatic rank determination candecomp parafac cp tensor factorization of incomplete data is a powerful technique for tensor completion through explicitly capturing the multilinear latent factors the existing cp algorithms require the tensor rank to be manually specified however the determination of tensor rank remains a challenging problem especially for cp rank in addition existing approaches do not take into account uncertainty information of latent factors as well as missing entries to address these issues we formulate cp factorization using a hierarchical probabilistic model and employ a fully bayesian treatment by incorporating a sparsity inducing prior over multiple latent factors and the appropriate hyperpriors over all hyperparameters resulting in automatic rank determination to learn the model we develop an efficient deterministic bayesian inference algorithm which scales linearly with data size our method is characterized as a tuning parameter free approach which can effectively infer underlying multilinear factors with a low rank constraint while also providing predictive distributions over missing entries extensive simulations on synthetic data illustrate the intrinsic capability of our method to recover the ground truth of cp rank and prevent the overfitting problem even when a large amount of entries are missing moreover the results from real world applications including image inpainting and facial image synthesis demonstrate that our method outperforms state of the art approaches for both tensor factorization and tensor completion in terms of predictive performance

the power of convex relaxation near optimal matrix completion this paper is concerned with the problem of recovering an unknown matrix from a small fraction of its entries this is known as the matrix completion problem and comes up in a great number of applications including the famous netflix prize and other similar questions in collaborative filtering in general accurate recovery of a matrix from a small number of entries is impossible but the knowledge that the unknown matrix has low rank radically changes this premise making the search for solutions meaningful this paper presents optimality results quantifying the minimum number of entries needed to recover a matrix of rank r exactly by any method whatsoever the information theoretic limit more importantly the paper shows that under certain incoherence assumptions on the singular vectors of the matrix recovery is possible by solving a convenient convex program as soon as the number of entries is on the order of the information theoretic limit up to logarithmic factors this convex program simply finds among all matrices consistent with the observed entries that with minimum nuclear norm as an example we show that on the order of nr log n samples are needed to recover a random n x n matrix of rank r by any method and to be sure nuclear norm minimization succeeds as soon as the number of entries is of the form nr polylog n

a generalized inverse method for asymptotic linear programming consider a linear program in which the entries of the coefficient matrix vary linearly with time to study the behavior of optimal solutions as time goes to infinity it is convenient to express the inverse of the basis matrix as a series expansion of powers of the time parameter we show that an algorithm of wilkinson 1982 for solving singular differential equations can be used to obtain such an expansion efficiently the resolvent expansions of dynamic programming are a special case of this method

a new robust d stability condition for real convex polytopic uncertainty abstract the problem of robust d stability analysis with respect to real convex polytopic uncertainties is tackled a new lmi based sufficient condition for the existence of parameter dependent lyapunov functions is proposed this condition generalises previously published conditions numerical comparisons with quadratic stability results as well as previous results based on parameter dependent lyapunov functions illustrate the relevance of this new condition finally this result appears to be promising for robust multi objective performance analysis and control synthesis purposes

laplacian eigenmaps for dimensionality reduction and data representation one of the central problems in machine learning and pattern recognition is to develop appropriate representations for complex data we consider the problem of constructing a representation for data lying on a low dimensional manifold embedded in a high dimensional space drawing on the correspondence between the graph laplacian the laplace beltrami operator on the manifold and the connections to the heat equation we propose a geometrically motivated algorithm for representing the high dimensional data the algorithm provides a computationally efficient approach to nonlinear dimensionality reduction that has locality preserving properties and a natural connection to clustering some potential applications and illustrative examples are discussed

an algorithm for computing a pade approximant with minimal degree denominator in this paper a new definition of a reduced pade approximant and an algorithm for its computation are proposed our approach is based on the investigation of the kernel structure of the toeplitz matrix it is shown that the reduced pade approximant always has nice properties which the classical pade approximant possesses only in the normal case the new algorithm allows us to avoid the appearance of froissart doublets induced by computer roundoff in the non normal pade table

an efficient basis update for asymptotic linear programming abstract for a linear program in which the constraint coefficients vary linearly with the time parameter we showed in a previous paper that a basic feasible solution can be evaluated using o k 1 m 3 arithmetic operations where m is the number of constraints and k is the index of the basis matrix pair here we show in the special case when k 1 for all basis matrix pairs and when one of the matrices in each pair has nearly full rank how the possibly singular matrix factorization can be updated with only o m 2 operations using rank one update techniques this makes the arithmetic complexity of updating the basis in asymptotic linear programming comparable to that of updating the inverse in ordinary linear programming in this case moreover we show that the result holds in particular when computing a blackwell optimal policy for markov decision chains in the unichain case or when all policies have only a small number of recurrent subchains

sparse non negative tensor factorization using columnwise coordinate descent many applications in computer vision biomedical informatics and graphics deal with data in the matrix or tensor form non negative matrix and tensor factorization which extract data dependent non negative basis functions have been commonly applied for the analysis of such data for data compression visualization and detection of hidden information factors in this paper we present a fast and flexible algorithm for sparse non negative tensor factorization sntf based on columnwise coordinate descent ccd different from the traditional coordinate descent which updates one element at a time ccd updates one column vector simultaneously our empirical results on higher mode images such as brain mri images gene expression images and hyperspectral images show that the proposed algorithm is 1 2 orders of magnitude faster than several state of the art algorithms

generalized discriminant analysis using a kernel approach we present a new method that we call generalized discriminant analysis gda to deal with nonlinear discriminant analysis using kernel function operator the underlying theory is close to the support vector machines svm insofar as the gda method provides a mapping of the input vectors into high dimensional feature space in the transformed space linear properties make it easy to extend and generalize the classical linear discriminant analysis lda to nonlinear discriminant analysis the formulation is expressed as an eigenvalue problem resolution using a different kernel one can cover a wide class of nonlinearities for both simulated data and alternate kernels we give classification results as well as the shape of the decision function the results are confirmed using real data to perform seed classification

perturbation of null spaces with application to the eigenvalue problem and generalized inverses we consider properties of a null space of an analytically perturbed matrix in particular we obtain taylor expansions for the eigenvectors which constitute a basis for the perturbed null space furthermore we apply these results to the calculation of puiseux expansion of the perturbed eigenvectors in the case of general eigenvalue problem as well as to the calculation of laurent series expansions for the perturbed group inverse and pseudoinverse matrices

some observations regarding interpolants in the limit of flat radial basis functions abstract radial basis functions rbfs form a primary tool for multivariate interpolation some of the most commonly used radial functions feature a shape parameter allowing them to vary from being nearly flat e small to sharply peaked e large the former limit can be particularly accurate when interpolating a smooth function based on scattered data this study discusses theoretical and computational aspects of the e 0 limit and includes the conjecture that gaussian rbf interpolants will never diverge in this limit

a reduction process for perturbed markov chains given a perturbed markov chain with generator a varepsilon a 0 varepsilon a 1 varepsilon 2 a 2 cdots it is shown that to each time scale t t varepsilon t varepsilon 2 cdots is associated a reduced markov chain this process of reduction has only finitely many steps it is given in a recursive way and each reduced chain admits as state space the recurrent classes of the reduced chain defined at the previous stage it is shown that if lambda is an eigenvalue different from zero of the generator of the kth reduced chain with multiplicity p then a varepsilon admits p eigenvalues lambda varepsilon such that lambda varepsilon varepsilon k lambda o varepsilon k a basis of ker a 0 resp ker a 0 which is robust to the perturbation is constructed

an lpd approach to robust h sub 2 and h sub spl infin static output feedback design a linear parameter dependent approach for designing a constant output feedback controller for a linear time invariant system with uncertain parameters that achieves a minimum bound on either the h sub 2 or h sub spl infin performance level is introduced assuming that the uncertain parameters reside in a given polytope a parameter dependent lyapunov function is described which enables the derivation of the required constant gain via a solution of a set of linear matrix inequalities that correspond to the vertices of the uncertainty polytope

limit problems for interpolation by analytic radial basis functions interpolation problems for analytic radial basis functions like the gaussian and inverse multiquadrics can degenerate in two ways the radial basis functions can be scaled to become increasingly flat or the data points coalesce in the limit while the radial basis functions stay fixed both cases call for a careful regularization which if carried out explicitly yields a preconditioning technique for the degenerating linear systems behind these interpolation problems this paper deals with both cases for the increasingly flat limit we recover results by larsson and fornberg together with lee yoon and yoon concerning convergence of interpolants towards polynomials with slight modifications the same technique can also handle scenarios with coalescing data points for fixed radial basis functions the results show that the degenerating local lagrange interpolation problems converge towards certain hermite birkhoff problems this is an important prerequisite for dealing with approximation by radial basis functions adaptively using freely varying data sites

mr ntd manifold regularization nonnegative tucker decomposition for tensor data dimension reduction and representation with the advancement of data acquisition techniques tensor multidimensional data objects are increasingly accumulated and generated for example multichannel electroencephalographies multiview images and videos in these applications the tensor objects are usually nonnegative since the physical signals are recorded as the dimensionality of tensor objects is often very high a dimension reduction technique becomes an important research topic of tensor data from the perspective of geometry high dimensional objects often reside in a low dimensional submanifold of the ambient space in this paper we propose a new approach to perform the dimension reduction for nonnegative tensor objects our idea is to use nonnegative tucker decomposition ntd to obtain a set of core tensors of smaller sizes by finding a common set of projection matrices for tensor objects to preserve geometric information in tensor data we employ a manifold regularization term for the core tensors constructed in the tucker decomposition an algorithm called manifold regularization ntd mr ntd is developed to solve the common projection matrices and core tensors in an alternating least squares manner the convergence of the proposed algorithm is shown and the computational complexity of the proposed method scales linearly with respect to the number of tensor objects and the size of the tensor objects respectively these theoretical results show that the proposed algorithm can be efficient extensive experimental results have been provided to further demonstrate the effectiveness and efficiency of the proposed mr ntd algorithm

sparse canonical temporal alignment with deep tensor decomposition for action recognition in this paper we solve three problems in action recognition sub action multi subject and multi modality by reducing the diversity of intra class samples the main stage contains canonical temporal alignment and key frames selection as we know temporal alignment aims to reduce the diversity of intra class samples however dense frames may yield misalignment or overlapped alignment and decrease recognition performance to overcome this problem we propose a sparse canonical temporal alignment scta method which selects and aligns key frames from two sequences to reduce diversity to extract better features from the key frames we propose a deep non negative tensor factorization dntf method to find a tensor subspace integrated with scta scheme first we model an action sequence as a third order tensor with spatiotemporal structure then we design a dntf scheme to find a tensor subspace in both spatial and temporal directions particularly in the first layer the original tensor is decomposed into two low rank tensors by ntf and in the second layer each low rank tensor is further decomposed by tensor train for time efficiency finally our framework composed of scta and dntf could solve the three problems and extract effective features for action recognition experiments on synthetic data msrdailyactivity3d and msractionpairs data sets show that our method works better than competitive methods in terms of accuracy

the parameter r2 in multiquadric interpolation the multiquadric mq method is an effective bivariate interpolant to three dimensional data xi yi zi where the xi yi are arbitrarily located in the plane the accuracy of the mq method is dependent on a user defined parameter r2 and most practitioners select r2 based upon the number of data points and the locations of the xi yi in the plane we observe that the optimal value of r2 is a strong function of the zi and that it is essentially independent of both the number and the locationsof the xi yi points contrary to some opinions we observe that the mq method can effectively interpolate to track data that is data sampled densely along tracks in the plane together with several other observations we present an algorithm that generally yields an effective value for r2

tensor distance based multilinear locality preserved maximum information embedding this brief paper presents a unified framework for tensor based dimensionality reduction dr with a new tensor distance td metric and a novel multilinear locality preserved maximum information embedding mlpmie algorithm different from traditional euclidean distance which is constrained by the orthogonality assumption td measures the distance between data points by considering the relationships among different coordinates to preserve the natural tensor structure in low dimensional space mlpmie directly works on the high order form of input data and iteratively learns the transformation matrices in order to preserve the local geometry and to maximize the global discrimination simultaneously mlpmie keeps both local and global structures in a manifold model by integrating td into tensor embedding td mlpmie performs tensor based dr through the whole learning procedure and achieves stable performance improvement on various standard datasets

probability guaranteed robust h performance analysis and state feedback design abstract this paper addresses the common engineering practice of specifying a required probability of attaining some performance level the problem setup is that of a robust h performance analysis state feedback synthesis of an affinely parameter dependent linear system except that the parameter hyper rectangle box is allowed to shrink representing a probability less than one in order to accommodate a polytopic performance goal that is better than the one attainable for the original parameter box a new version of the bounded real lemma brl which assigns a different lyapunov function to each of the vertices of the uncertainty polytope and includes a scalar free parameter seems to reduce the overdesign associated with the polytopic problem to the bare minimum the shrinking of the parameter box leads to bi linear matrix inequalities blmis since the final vertices are also unknown these blmis are solved iteratively three steps have sufficed both in the analysis and in the state feedback design examples a uniform distribution is assumed for all the system parameters following the uniformity principle the probability requirement is expressed by a set of linear matrix inequalities lmis that is derived by extending an existing second order cone method these lmis are concurrently solved with the blmis of the brl the features of the proposed method are demonstrated via two examples

probabilistic robust design with linear quadratic regulators in this paper we study robust design of uncertain systems in a probabilistic setting by means of linear quadratic regulators lqr we consider systems affected by random bounded nonlinear uncertainty so that classical optimization methods based on linear matrix inequalities cannot be used without conservatism the approach followed here is a blend of randomization techniques for the uncertainty together with convex optimization for the controller parameters in particular we propose an iterative algorithm for designing a controller which is based upon subgradient iterations at each step of the sequence we first generate a random sample and then we perform a subgradient step for a convex constraint defined by the lqr problem the main result of the paper is to prove that this iterative algorithm provides a controller which quadratically stabilizes the uncertain system with probability one in a finite number of steps in addition at a fixed step we compute a lower bound of the probability that a quadratically stabilizing controller is found