the matrix exponential there are many different methods to calculate the exponential of a matrix series methods differential equations methods polynomial methods matrix decomposition methods and splitting methods none of which is entirely satisfactory from either a theoretical or a computational point of view how then should the matrix exponential be introduced in an elementary differential equations course for engineering students for example with a minimum of mathematical prerequisites in this note a method is given that uses the students knowledge of homogeneous linear differential equations with constant coefficients and the cayley hamilton theorem the method is not new what is new is the approach

multiple kernel learning for dimensionality reduction in solving complex visual learning tasks adopting multiple descriptors to more precisely characterize the data has been a feasible way for improving performance the resulting data representations are typically high dimensional and assume diverse forms hence finding a way of transforming them into a unified space of lower dimension generally facilitates the underlying tasks such as object recognition or clustering to this end the proposed approach termed mkl dr generalizes the framework of multiple kernel learning for dimensionality reduction and distinguishes itself with the following three main contributions first our method provides the convenience of using diverse image descriptors to describe useful characteristics of various aspects about the underlying data second it extends a broad set of existing dimensionality reduction techniques to consider multiple kernel learning and consequently improves their effectiveness third by focusing on the techniques pertaining to dimensionality reduction the formulation introduces a new class of applications with the multiple kernel learning framework to address not only the supervised learning problems but also the unsupervised and semi supervised ones

mixture subclass discriminant analysis link to restricted gaussian model and other generalizations in this paper a theoretical link between mixture subclass discriminant analysis msda and a restricted gaussian model is first presented then two further discriminant analysis da methods i e fractional step msda fsmsda and kernel msda kmsda are proposed linking msda to an appropriate gaussian model allows the derivation of a new da method under the expectation maximization em framework em msda which simultaneously derives the discriminant subspace and the maximum likelihood estimates the two other proposed methods generalize msda in order to solve problems inherited from conventional da fsmsda solves the subclass separation problem that is the situation in which the dimensionality of the discriminant subspace is strictly smaller than the rank of the inter between subclass scatter matrix this is done by an appropriate weighting scheme and the utilization of an iterative algorithm for preserving useful discriminant directions on the other hand kmsda uses the kernel trick to separate data with nonlinearly separable subclass structure extensive experimentation shows that the proposed methods outperform conventional msda and other linear discriminant analysis variants

multi element generalized polynomial chaos for arbitrary probability measures we develop a multi element generalized polynomial chaos me gpc method for arbitrary probability measures and apply it to solve ordinary and partial differential equations with stochastic inputs given a stochastic input with an arbitrary probability measure its random space is decomposed into smaller elements subsequently in each element a new random variable with respect to a conditional probability density function pdf is defined and a set of orthogonal polynomials in terms of this random variable is constructed numerically then the generalized polynomial chaos gpc method is implemented element by element numerical experiments show that the cost for the construction of orthogonal polynomials is negligible compared to the total time cost efficiency and convergence of me gpc are studied numerically by considering some commonly used random variables me gpc provides an efficient and flexible approach to solving differential equations with random inputs especially for problems related to long term integration large perturbation and stochastic discontinuities

dynamic least squares kernel density modeling of fokker planck equations with application to neural population the dynamic least squares kernel density lsqkd model c pantano and b shotorban phys rev e 76 066705 2007 is used to solve the fokker planck equations in this model the probability density function pdf is approximated by a linear combination of basis functions with unknown parameters whose governing equations are determined by a global least squares approximation of the pdf in the phase space in this work basis functions are set to be gaussian for which the mean variance and covariances are governed by a set of partial differential equations pdes or ordinary differential equations odes depending on what phase space variables are approximated by gaussian functions three sample problems of univariate double well potential bivariate bistable neurodynamical system g deco and d mart i phys rev e 75 031913 2007 and bivariate brownian particles in a nonuniform gas are studied the lsqkd is verified for these problems as its results are compared against the results of the method of characteristics in nondiffusive cases and the stochastic particle method in diffusive cases for the double well potential problem it is observed that for low to moderate diffusivity the dynamic lsqkd well predicts the stationary pdf for which there is an exact solution a similar observation is made for the bistable neurodynamical system in both these problems least squares approximation is made on all phase space variables resulting in a set of odes with time as the independent variable for the gaussian function parameters in the problem of brownian particles in a nonuniform gas this approximation is made only for the particle velocity variable leading to a set of pdes with time and particle position as independent variables solving these pdes a very good performance by lsqkd is observed for a wide range of diffusivities

rotation invariant kernels and their application to shape analysis shape analysis requires invariance under translation scale and rotation translation and scale invariance can be realized by normalizing shape vectors with respect to their mean and norm this maps the shape feature vectors onto the surface of a hypersphere after normalization the shape vectors can be made rotational invariant by modeling the resulting data using complex scalar rotation invariant distributions defined on the complex hypersphere e g using the complex bingham distribution however the use of these distributions is hampered by the difficulty in estimating their parameters and the nonlinear nature of their formulation in the present paper we show how a set of kernel functions that we refer to as rotation invariant kernels can be used to convert the original nonlinear problem into a linear one as their name implies these kernels are defined to provide the much needed rotation invariance property allowing one to bypass the difficulty of working with complex spherical distributions the resulting approach provides an easy fast mechanism for 2d 3d shape analysis extensive validation using a variety of shape modeling and classification problems demonstrates the accuracy of this proposed approach

pruning noisy bases in discriminant analysis the success of linear discriminant analysis lda is due in part to the simplicity of its formulation which reduces to a simultaneous diagonalization of two symmetric matrices a and b however a fundamental drawback of this approach is that it cannot be efficiently applied wherever the matrix a is singular or when some of the smallest variances in are due to noise in this paper we present a factorization a 1b of and a correlation based criterion that can be readily employed to solve these problems we provide detailed derivations for the linear and nonlinear classification problems the usefulness of the proposed approach is demonstrated thoroughly using a large variety of databases

minimizing nearest neighbor classification error for nonparametric dimension reduction in this brief we show that minimizing nearest neighbor classification error mnne is a favorable criterion for supervised linear dimension reduction sldr we prove that mnne is better than maximizing mutual information in the sense of being a proxy of the bayes optimal criterion based on kernel density estimation we derive a nonparametric algorithm for mnne experiments on benchmark data sets show the superiority of mnne over existing nonparametric sldr methods

multi resolution analysis of wiener type uncertainty propagation schemes a multi resolution analysis mra is applied to an uncertainty propagation scheme based on a generalized polynomial chaos pc representation the mra relies on an orthogonal projection of uncertain data and solution variables onto a multi wavelet basis consisting of compact piecewise smooth polynomial functions the coefficients of the expansion are computed through a galerkin procedure the mra scheme is applied to the simulation of the lorenz system having a single random parameter the convergence of the solution with respect to the resolution level and expansion order is investigated in particular results are compared to two monte carlo sampling strategies demonstrating the superiority of the mra for more complex problems however the mra approach may require excessive cpu times adaptive methods are consequently developed in order to overcome this drawback two approaches are explored the first is based on adaptive refinement of the multi wavelet basis while the second is based on adaptive block partitioning of the space of random variables computational tests indicate that the latter approach is better suited for large problems leading to a more efficient flexible and parallelizable scheme

learning linear discriminant projections for dimensionality reduction of image descriptors in this paper we present linear discriminant projections ldp for reducing dimensionality and improving discriminability of local image descriptors we place ldp into the context of state of the art discriminant projections and analyze its properties ldp requires a large set of training data with point to point correspondence ground truth we demonstrate that training data produced by a simulation of image transformations leads to nearly the same results as the real data with correspondence ground truth this makes it possible to apply ldp as well as other discriminant projection approaches to the problems where the correspondence ground truth is not available such as image categorization we perform an extensive experimental evaluation on standard data sets in the context of image matching and categorization we demonstrate that ldp enables significant dimensionality reduction of local descriptors and performance increases in different applications the results improve upon the state of the art recognition performance with simultaneous dimensionality reduction from 128 to 30

multi class discriminant kernel learning via convex programming regularized kernel discriminant analysis rkda performs linear discriminant analysis in the feature space via the kernel trick its performance depends on the selection of kernels in this paper we consider the problem of multiple kernel learning mkl for rkda in which the optimal kernel matrix is obtained as a linear combination of pre specified kernel matrices we show that the kernel learning problem in rkda can be formulated as convex programs first we show that this problem can be formulated as a semidefinite program sdp based on the equivalence relationship between rkda and least square problems in the binary class case we propose a convex quadratically constrained quadratic programming qcqp formulation for kernel learning in rkda a semi infinite linear programming silp formulation is derived to further improve the efficiency we extend these formulations to the multi class case based on a key result established in this paper that is the multi class rkda kernel learning problem can be decomposed into a set of binary class kernel learning problems which are constrained to share a common kernel based on this decomposition property sdp formulations are proposed for the multi class case furthermore it leads naturally to qcqp and silp formulations as the performance of rkda depends on the regularization parameter we show that this parameter can also be optimized in a joint framework with the kernel extensive experiments have been conducted and analyzed and connections to other algorithms are discussed

regularized discriminant analysis abstract linear and quadratic discriminant analysis are considered in the small sample high dimensional setting alternatives to the usual maximum likelihood plug in estimates for the covariance matrices are proposed these alternatives are characterized by two parameters the values of which are customized to individual situations by jointly minimizing a sample based estimate of future misclassification risk computationally fast implementations are presented and the efficacy of the approach is examined through simulation studies and application to data these studies indicate that in many circumstances dramatic gains in classification accuracy can be achieved

graph embedding and extensions a general framework for dimensionality reduction a large family of algorithms supervised or unsupervised stemming from statistics or geometry theory has been designed to provide different solutions to the problem of dimensionality reduction despite the different motivations of these algorithms we present in this paper a general formulation known as graph embedding to unify them within a common framework in graph embedding each algorithm can be considered as the direct graph embedding or its linear kernel tensor extension of a specific intrinsic graph that describes certain desired statistical or geometric properties of a data set with constraints from scale normalization or a penalty graph that characterizes a statistical or geometric property that should be avoided furthermore the graph embedding framework can be used as a general platform for developing new dimensionality reduction algorithms by utilizing this framework as a tool we propose a new supervised dimensionality reduction algorithm called marginal fisher analysis in which the intrinsic graph characterizes the intraclass compactness and connects each data point with its neighboring points of the same class while the penalty graph connects the marginal points and characterizes the interclass separability we show that mfa effectively overcomes the limitations of the traditional linear discriminant analysis algorithm due to data distribution assumptions and available projection directions real face recognition experiments show the superiority of our proposed mfa in comparison to lda also for corresponding kernel and tensor extensions

efficient learning of supervised kernels with a graph based loss function this paper presents our study of the problems associated with learning supervised kernels from a large amount of side information we propose a new loss function derived from the laplacian matrix of a special complete graph that is generated from the side information we analyze the relationship between the proposed loss function and the kernel alignment our theoretical analysis shows that the proposed loss function has a close relationship with kernel alignment that is they both make use of side information that is fused in a matrix in addition to a similar regularization strategy moreover the proposed loss function has a linear form and thus it is more efficient in learning side information than kernel alignment that has to be performed nonlinearly the proposed loss function is used to generate new kernels as low cost alternatives of kernels learned by certain state of the art methods the empirical results demonstrate the superiority of the proposed method over state of the art methods in terms of classification accuracy and computational cost

improved data visualisation through nonlinear dissimilarity modelling inherent to state of the art dimension reduction algorithms is the assumption that global distances between observations are euclidean despite the potential for altogether non euclidean data manifolds we demonstrate that a non euclidean manifold chart can be approximated by implementing a universal approximator over a dictionary of dissimilarity measures building on recent developments in the field this approach is transferable across domains such that observations can be vectors distributions graphs and time series for instance our novel dissimilarity learning method is illustrated with four standard visualisation datasets showing the benefits over the linear dissimilarity learning approach

semantic preserving distance metric learning and applications how do we accurately browse a large set of images or efficiently annotate the images from an image library image clustering methods are invaluable tools for applications such as content based image retrieval and image annotation to perform these tasks it is critical to have proper features to describe the visual and semantic content of images and to define an accurate distance metric to measure the dissimilarity between any two images however existing methods which adopt the features of color histograms edge direction histograms and shape context lack the ability to describe semantic content to solve this problem we propose a new approach that utilizes user provided pairwise constraints to describe the semantic relationship between two images a semantic preserving distance metric learning sp dml algorithm is developed to explore the complementary characteristics of the visual features and pairwise constraints in a unified feature space in this space the learned distance metric can be used to measure the dissimilarity between two images specifically the manifold structure adopted in sp dml is revealed by the image s visual features to integrate semantic contents in distance metric learning sp dml utilizes pairwise constraints to build semantic patches and align these patches to obtain the optimal distance metric for the new feature space experimental results in image clustering demonstrate that the performance of sp dml is appealing

a multi view embedding space for modeling internet images tags and their semantics this paper investigates the problem of modeling internet images and associated text or tags for tasks such as image to image search tag to image search and image to tag search image annotation we start with canonical correlation analysis cca a popular and successful approach for mapping visual and textual features to the same latent space and incorporate a third view capturing high level image semantics represented either by a single category or multiple non mutually exclusive concepts we present two ways to train the three view embedding supervised with the third view coming from ground truth labels or search keywords and unsupervised with semantic themes automatically obtained by clustering the tags to ensure high accuracy for retrieval tasks while keeping the learning process scalable we combine multiple strong visual features and use explicit nonlinear kernel mappings to efficiently approximate kernel cca to perform retrieval we use a specially designed similarity function in the embedded space which substantially outperforms the euclidean distance the resulting system produces compelling qualitative results and outperforms a number of two view baselines on retrieval tasks on three large scale internet image datasets

interpolating distributed approximating functionals in this paper we present a class of distributed approximating functionals daf s for solving various problems in the sciences and engineering previous daf s were specifically constructed to avoid interpolation in order to achieve the well tempered limit in which the same order error is made both on and off the grid points these daf s are constructed by combining the daf concept with various interpolation schemes the approach then becomes the same as the moving least squares method but the specific interpolating daf s obtained are new to our knowledge these interpolating daf s are illustrated using lagrange interpolation the ldaf and a gaussian weight function four numerical tests are used to illustrate the ldaf s differentiation on and off a grid fitting a function off a grid time dependent quantum dynamical evolution and solving nonlinear burgers equation s1063 651x 98 10204 0

kernel optimization in discriminant analysis kernel mapping is one of the most used approaches to intrinsically derive nonlinear classifiers the idea is to use a kernel function which maps the original nonlinearly separable problem to a space of intrinsically larger dimensionality where the classes are linearly separable a major problem in the design of kernel methods is to find the kernel parameters that make the problem linear in the mapped representation this paper derives the first criterion that specifically aims to find a kernel representation where the bayes classifier becomes linear we illustrate how this result can be successfully applied in several kernel discriminant analysis algorithms experimental results using a large number of databases and classifiers demonstrate the utility of the proposed approach the paper also shows theoretically and experimentally that a kernel version of subclass discriminant analysis yields the highest recognition rates

mean vector component analysis for visualization and clustering of nonnegative data mean vector component analysis mvca is introduced as a new method for visualization and clustering of nonnegative data the method is based on dimensionality reduction by preserving the squared length and implicitly also the direction of the mean vector of the original data the optimal mean vector preserving basis is obtained from the spectral decomposition of the inner product matrix and it is shown to capture clustering structure mvca corresponds to certain uncentered principal component analysis pca axes unlike traditional pca these axes are in general not corresponding to the top eigenvalues mvca is shown to produce different visualizations and sometimes considerably improved clustering results for nonnegative data compared with pca

least squares dynamic approximation method for evolution of uncertainty in initial conditions of dynamical systems we describe an approximation method to solve the probability density function transport equation i e the liouville equation which is encountered in the evolution of uncertainty of the initial values of dynamical systems a state space based method is formulated using a least squares technique that preserves the parabolic nature of the liouville equation and is flexible in terms of accuracy of representation this method is based on a global approximation in terms of analytical elementary functions with unknown parameters whose evolution equations are determined by a global least squares approximation the realizability conditions of the probability density i e the non negativity and normalization conditions are enforced at all times the method is successfully evaluated in a number of scenarios including the uncertainty evolution in a system governed by a riccati equation and a particle moving in a fluid under the influence of stokes drag force the results obtained in our examples exhibit a reasonable good agreement when compared with the solution of the probability transport equation using the method of characteristics the cost of the method is proportional to the cost of solving the deterministic system and the number of parameters used to approximate the probability density function a feature that can make the present method very advantageous in comparison with other methods in problems involving a large number of dimensions

the wiener askey polynomial chaos for stochastic differential equations we present a new method for solving stochastic differential equations based on galerkin projections and extensions of wiener s polynomial chaos specifically we represent the stochastic processes with an optimum trial basis from the askey family of orthogonal polynomials that reduces the dimensionality of the system and leads to exponential convergence of the error several continuous and discrete processes are treated and numerical examples show substantial speed up compared to monte carlo simulations for low dimensional stochastic inputs

kpca plus lda a complete kernel fisher discriminant framework for feature extraction and recognition this paper examines the theory of kernel fisher discriminant analysis kfd in a hilbert space and develops a two phase kfd framework i e kernel principal component analysis kpca plus fisher linear discriminant analysis lda this framework provides novel insights into the nature of kfd based on this framework the authors propose a complete kernel fisher discriminant analysis ckfd algorithm ckfd can be used to carry out discriminant analysis in double discriminant subspaces the fact that it can make full use of two kinds of discriminant information regular and irregular makes ckfd a more powerful discriminator the proposed algorithm was tested and evaluated using the feret face database and the cenparmi handwritten numeral database the experimental results show that ckfd outperforms other kfd algorithms

dimensionality reduction of multimodal labeled data by local fisher discriminant analysis reducing the dimensionality of data without losing intrinsic information is an important preprocessing step in high dimensional data analysis fisher discriminant analysis fda is a traditional technique for supervised dimensionality reduction but it tends to give undesired results if samples in a class are multimodal an unsupervised dimensionality reduction method called locality preserving projection lpp can work well with multimodal data due to its locality preserving property however since lpp does not take the label information into account it is not necessarily useful in supervised learning scenarios in this paper we propose a new linear supervised dimensionality reduction method called local fisher discriminant analysis lfda which effectively combines the ideas of fda and lpp lfda has an analytic form of the embedding transformation and the solution can be easily computed just by solving a generalized eigenvalue problem we demonstrate the practical usefulness and high scalability of the lfda method in data visualization and classification tasks through extensive simulation studies we also show that lfda can be extended to non linear dimensionality reduction scenarios by applying the kernel trick

bayes optimality in linear discriminant analysis we present an algorithm that provides the one dimensional subspace where the bayes error is minimized for the c class problem with homoscedastic gaussian distributions our main result shows that the set of possible one dimensional spaces v for which the order of the projected class means is identical defines a convex region with associated convex bayes error function g v this allows for the minimization of the error function using standard convex optimization algorithms our algorithm is then extended to the minimization of the bayes error in the more general case of heteroscedastic distributions this is done by means of an appropriate kernel mapping function this result is further extended to obtain the d dimensional solution for any given d by iteratively applying our algorithm to the null space of the d l dimensional solution we also show how this result can be used to improve upon the outcomes provided by existing algorithms and derive a low computational cost linear approximation extensive experimental validations are provided to demonstrate the use of these algorithms in classification data analysis and visualization

mreklm a fast multiple empirical kernel learning machine abstract multiple empirical kernel learning mekl explicitly maps samples into different empirical feature spaces in which the kernel features of the mapped samples can be directly provided thus mekl is much easier than the conventional multiple kernel learning mkl in terms of processing and analyzing the structure of mapped feature spaces however the computational complexity of mekl with m empirical feature spaces is o mn 3 where n is the number of training samples the dimensions of the generated empirical feature spaces are approximate to n when dealing with large scale problems mekl cannot handle them properly due to the severe computation and memory burden moreover most existing mekl utilizes the gradient decent optimization to learn classifiers but it is time consuming for training therefore this paper proposes a multiple random empirical kernel learning machine mreklm to overcome these problems the proposed mreklm adopts the random projection idea to map samples into multiple low dimensional empirical feature spaces with lower computational complexity o mp 3 where p n is the number of the randomly selected samples after that mreklm adopts an analytical optimization approach to directly deal with multi class problems the computational complexity of mreklm is o m 3 p 3 experimental results also validate both efficiency and effectiveness of the proposed mreklm the contributions of this work are 1 proposing a fast mekl algorithm named mreklm 2 introducing an efficient random empirical kernel mapping approach and 3 extending the capability of mekl to handle large scale problems

stability radii of higher order positive difference systems in this paper we study stability radii of positive polynomial matrices under affine perturbations of the coefficient matrices it is shown that the real and complex stability radii coincide moreover explicit formulas are derived for these stability radii and illustrated by some examples

subclass discriminant analysis over the years many discriminant analysis da algorithms have been proposed for the study of high dimensional data in a large variety of problems each of these algorithms is tuned to a specific type of data distribution that which best models the problem at hand unfortunately in most problems the form of each class pdf is a priori unknown and the selection of the da algorithm that best fits our data is done over trial and error ideally one would like to have a single formulation which can be used for most distribution types this can be achieved by approximating the underlying distribution of each class with a mixture of gaussians in this approach the major problem to be addressed is that of determining the optimal number of gaussians per class i e the number of subclasses in this paper two criteria able to find the most convenient division of each class into a set of subclasses are derived extensive experimental results are shown using five databases comparisons are given against linear discriminant analysis lda direct lda dlda heteroscedastic lda hlda nonparametric da nda and kernel based lda k lda we show that our method is always the best or comparable to the best

a fast solver for fokker planck equation applied to viscoelastic flows calculations 2d fene model in this paper we introduce a new method for solving efficiently the fokker planck equation arising in the simulation of dilute polymeric solutions for a two dimensional fene finitely extensible nonlinear elastic model the structure of the fokker planck equation is used to design a fast solver the resulting equations are discretized with a spectral spectral element method application of the method to flow of a fene fluid past a confined cylinder demonstrates readily the advantages of the proposed scheme over traditional stochastic simulations

new evolution equations for the joint response excitation probability density function of stochastic solutions to first order nonlinear pdes by using functional integral methods we determine new evolution equations satisfied by the joint response excitation probability density function pdf associated with the stochastic solution to first order nonlinear partial differential equations pdes the theory is presented for both fully nonlinear and for quasilinear scalar pdes subject to random boundary conditions random initial conditions or random forcing terms particular applications are discussed for the classical linear and nonlinear advection equations and for the advection reaction equation by using a fourier galerkin spectral method we obtain numerical solutions of the proposed response excitation pdf equations these numerical solutions are compared against those obtained by using more conventional statistical approaches such as probabilistic collocation and multi element probabilistic collocation methods it is found that the response excitation approach yields accurate predictions of the statistical properties of the system in addition it allows to directly ascertain the tails of probabilistic distributions thus facilitating the assessment of rare events and associated risks the computational cost of the response excitation method is order magnitudes smaller than the one of more conventional statistical approaches if the pde is subject to high dimensional random boundary or initial conditions the question of high dimensionality for evolution equations involving multidimensional joint response excitation pdfs is also addressed

geometry based ensembles toward a structural characterization of the classification boundary this article introduces a novel binary discriminative learning technique based on the approximation of the non linear decision boundary by a piece wise linear smooth additive model the decision border is geometrically defined by means of the characterizing boundary points points that belong to the optimal boundary under a certain notion of robustness based on these points a set of locally robust linear classifiers is defined and assembled by means of a tikhonov regularized optimization procedure in an additive model to create a final lambda smooth decision rule as a result a very simple and robust classifier with a strong geometrical meaning and non linear behavior is obtained the simplicity of the method allows its extension to cope with some of nowadays machine learning challenges such as online learning large scale learning or parallelization with linear computational complexity we validate our approach on the uci database finally we apply our technique in online and large scale scenarios and in six real life computer vision and pattern recognition problems gender recognition intravascular ultrasound tissue classification speed traffic sign detection chagas disease severity detection clef classification and action recognition using a 3d accelerometer data the results are promising and this paper opens a line of research that deserves further attention

multiclass linear dimension reduction by weighted pairwise fisher criteria we derive a class of computationally inexpensive linear dimension reduction criteria by introducing a weighted variant of the well known k class fisher criterion associated with linear discriminant analysis lda it can be seen that lda weights contributions of individual class pairs according to the euclidean distance of the respective class means we generalize upon lda by introducing a different weighting function

on formal solutions of linear matrix differential difference equations abstract in this paper we construct a formal solution to the matrix differential difference equation dde y t a t y t 1 where a t is a matrix power series in t 1 in many cases solutions to the latter equation and to the matrix differential equation y t a t y t have the same form however these solutions may have different forms when the spectrum of a 0 the leading term of a t contains e 1

the truncated stieltjes moment problem solved by using kernel density functions in this work the problem of the approximate numerical determination of a semi infinite supported continuous probability density function pdf from a finite number of its moments is addressed the target space is carefully defined and an approximation theorem is proved establishing that the set of all convex superpositions of appropriate kernel density functions kdfs is dense in this space a solution algorithm is provided based on the established approximate representation of the target pdf and the exploitation of some theoretical results concerning moment sequence asymptotics the solution algorithm also permits us to recover the tail behavior of the target pdf and incorporate this information in our solution a parsimonious formulation of the proposed solution procedure based on a novel sequentially adaptive scheme is developed enabling a very efficient moment data inversion the whole methodology is fully illustrated by numerical examples

generalized discriminant analysis using a kernel approach we present a new method that we call generalized discriminant analysis gda to deal with nonlinear discriminant analysis using kernel function operator the underlying theory is close to the support vector machines svm insofar as the gda method provides a mapping of the input vectors into high dimensional feature space in the transformed space linear properties make it easy to extend and generalize the classical linear discriminant analysis lda to nonlinear discriminant analysis the formulation is expressed as an eigenvalue problem resolution using a different kernel one can cover a wide class of nonlinearities for both simulated data and alternate kernels we give classification results as well as the shape of the decision function the results are confirmed using real data to perform seed classification

large margin multi viewinformation bottleneck in this paper we extend the theory of the information bottleneck ib to learning from examples represented by multi view features we formulate the problem as one of encoding a communication system with multiple senders each of which represents one view of the data based on the precise components filtered out from multiple information sources through a bottleneck a margin maximization approach is then used to strengthen the discrimination of the encoder by improving the code distance within the frame of coding theory the resulting algorithm therefore inherits all the merits of the ib principle and coding theory it has two distinct advantages over existing algorithms namely that our method finds a tradeoff between the accuracy and complexity of the multi view model and that the encoded multi view data retains sufficient discrimination for classification we also derive the robustness and generalization error bound of the proposed algorithm and reveal the specific properties of multi view learning first the complementarity of multi view features guarantees the robustness of the algorithm second the consensus of multi view features reduces the empirical rademacher complexity of the objective function enhances the accuracy of the solution and improves the generalization error bound of the algorithm the resulting objective function is solved efficiently using the alternating direction method experimental results on annotation classification and recognition tasks demonstrate that the proposed algorithm is promising for practical applications

multi kernel learning with online batch optimization in recent years there has been a lot of interest in designing principled classification algorithms over multiple cues based on the intuitive notion that using more features should lead to better performance in the domain of kernel methods a principled way to use multiple features is the multi kernel learning mkl approach here we present a mkl optimization algorithm based on stochastic gradient descent that has a guaranteed convergence rate we directly solve the mkl problem in the primal formulation by having a p norm formulation of mkl we introduce a parameter that controls the level of sparsity of the solution while leading to an easier optimization problem we prove theoretically and experimentally that 1 our algorithm has a faster convergence rate as the number of kernels grows 2 the training complexity is linear in the number of training examples 3 very few iterations are sufficient to reach good solutions experiments on standard benchmark databases support our claims

nonparametric discriminant analysis a nonparametric method of discriminant analysis is proposed it is based on nonparametric extensions of commonly used scatter matrices two advantages result from the use of the proposed nonparametric scatter matrices first they are generally of full rank this provides the ability to specify the number of extracted features desired this is in contrast to parametric discriminant analysis which for an l class problem typically can determine at most l 1 features second the nonparametric nature of the scatter matrices allows the procedure to work well even for non gaussian data sets using the same basic framework a procedure is proposed to test the structural similarity of two distributions the procedure works in high dimensional space it specifies a linear decomposition of the original data space in which a relative indication of dissimilarity along each new basis vector is provided the nonparametric scatter matrices are also used to derive a clustering procedure which is recognized as a k nearest neighbor version of the nonparametric valley seeking algorithm the form which results provides a unified view of the parametric nearest mean reclassification algorithm and the nonparametric valley seeking algorithm

linear dimensionality reduction via a heteroscedastic extension of lda the chernoff criterion we propose an eigenvector based heteroscedastic linear dimension reduction ldr technique for multiclass data the technique is based on a heteroscedastic two class technique which utilizes the so called chernoff criterion and successfully extends the well known linear discriminant analysis lda the latter which is based on the fisher criterion is incapable of dealing with heteroscedastic data in a proper way for the two class case the between class scatter is generalized so to capture differences in co variances it is shown that the classical notion of between class scatter can be associated with euclidean distances between class means from this viewpoint the between class scatter is generalized by employing the chernoff distance measure leading to our proposed heteroscedastic measure finally using the results from the two class case a multiclass extension of the chernoff criterion is proposed this criterion combines separation information present in the class mean as well as the class covariance matrices extensive experiments and a comparison with similar dimension reduction techniques are presented

a stochastic projection method for fluid flow i basic formulation we describe the construction and implementation of a stochastic navier stokes solver the solver combines a spectral stochastic uncertainty representation scheme with a finite difference projection method for flow simulation the uncertainty quantification scheme is adapted from the spectral stochastic finite element method ssfem which is based on regarding uncertainty as generating a new dimension and the solution as being dependent on this dimension in the ssfem formalism the stochastic dependence is represented in terms of the polynomial chaos system and the coefficients in the corresponding spectral representation are obtained using a galerkin approach it is shown that incorporation of the spectral uncertainty representation scheme into the projection method results in a coupled system of advection diffusion equations for the various uncertainty fields and in a decoupled system of pressure projection steps this leads to a very efficient stochastic solver whose advantages are illustrated using steady and transient simulations of transport and mixing in a microchannel

two stage multiple kernel learning for supervised dimensionality reduction in supervised dimensionality reduction methods for pattern recognition tasks the information of the class labels is considered through the process of reducing the input dimensionality to improve the classification accuracy using nonlinear mappings for this purpose makes these models more appropriate for nonlinearly distributed data in this paper a new nonlinear supervised dimensionality reduction model is introduced the dimensionality reduction process in this model is performed through a multiple kernel learning paradigm in two stages in the first stage three suitable criteria for supervised dimensionality reduction containing fisher homoscedasticity and between class distance criteria are used to find the kernel weights with these weights a linear combination of several valid kernels is computed to make a new suitable kernel function in the second stage the kernel discriminant analysis method is employed for nonlinear supervised dimensionality reduction using the kernel computed in the first stage many experiments on a variety of real world datasets including handwritten digits images objects images and other datasets show that the proposed approach among a number of well known related techniques results in accurate and fast classifications a novel approach for non linear supervised dimensionality reduction is proposed the proposed method uses kernel discriminant analysis method with new kernels a multiple kernel learning mkl paradigm produces the new kernel function proper criteria for supervised dimensionality reduction are optimized in the mkl part the proposed method is successfully evaluated using many real world datasets

a kernel induced space selection approach to model selection in klda model selection in kernel linear discriminant analysis klda refers to the selection of appropriate parameters of a kernel function and the regularizer by following the principle of maximum information preservation this paper formulates the model selection problem as a problem of selecting an optimal kernel induced space in which different classes are maximally separated from each other a scatter matrix based criterion is developed to measure the goodness of a kernel induced space and the kernel parameters are tuned by maximizing this criterion this criterion is computationally efficient and is differentiable with respect to the kernel parameters compared with the leave one out loo or k fold cross validation cv the proposed approach can achieve a faster model selection especially when the number of training samples is large or when many kernel parameters need to be tuned to tune the regularization parameter in the klda our criterion is used together with the method proposed by saadi 2004 experiments on benchmark data sets verify the effectiveness of this model selection approach

on the computation of the jordan canonical form of regular matrix polynomials in this paper an algorithm for the computation of the jordan canonical form of regular matrix polynomials is proposed the new method contains rank conditions of suitably defined block toeplitz matrices and it does not require the computation of the jordan chains or the smith form the segre and weyr characteristics are also considered

semisupervised multiview distance metric learning for cartoon synthesis in image processing cartoon character classification retrieval and synthesis are critical so that cartoonists can effectively and efficiently make cartoons by reusing existing cartoon data to successfully achieve these tasks it is essential to extract visual features that comprehensively represent cartoon characters and to construct an accurate distance metric to precisely measure the dissimilarities between cartoon characters in this paper we introduce three visual features color histogram shape context and skeleton to characterize the color shape and action respectively of a cartoon character these three features are complementary to each other and each feature set is regarded as a single view however it is improper to concatenate these three features into a long vector because they have different physical properties and simply concatenating them into a high dimensional feature vector will suffer from the so called curse of dimensionality hence we propose a semisupervised multiview distance metric learning ssm dml ssm dml learns the multiview distance metrics from multiple feature sets and from the labels of unlabeled cartoon characters simultaneously under the umbrella of graph based semisupervised learning ssm dml discovers complementary characteristics of different feature sets through an alternating optimization based iterative algorithm therefore ssm dml can simultaneously accomplish cartoon character classification and dissimilarity measurement on the basis of ssm dml we develop a novel system that composes the modules of multiview cartoon character classification multiview graph based cartoon synthesis and multiview retrieval based cartoon synthesis experimental evaluations based on the three modules suggest the effectiveness of ssm dml in cartoon applications

on non homogeneous singular systems of fractional nabla difference equations in this article we study the initial value problem of a class of non homogeneous singular systems of fractional nabla difference equations whose coefficients are constant matrices by taking into consideration the cases that the matrices are square with the leading coefficient singular non square and square with a matrix pencil which has an identically zero determinant we provide necessary and sufficient conditions for the existence and uniqueness of solutions more analytically we study the conditions under which the system has unique infinite and no solutions furthermore we provide a formula for the case of the unique solution finally numerical examples are given to justify our theory

perturbation and robust stability of autonomous singular linear matrix difference equations in the perturbation theory of linear matrix difference equations it is well known that the theory of finite and infinite elementary divisors of regular matrix pencils is complicated by the fact that arbitrarily small perturbations of the pencil can cause them to disappear in this paper the perturbation theory of complex weierstrass canonical form for regular matrix pencils is investigated by using matrix pencil theory and the weierstrass canonical form of the pencil we obtain bounds for the finite elementary divisors of a perturbed pencil moreover we study robust stability of a class of linear matrix difference equations of first and higher order whose coefficients are square constant matrices