an adaptive direct variational grid generation method abstract variational grid generation techniques are now used to produce grids suitable for solving numerical partial differential equations in irregular geometries variational grid generation methods are very robust but slow the method considered here is a discrete variational method this method preserves the robustness of the variational method also it is very fast so it can be used in time dependent problems here we discuss geometry and solution adaption for the direct discrete grid generation method and some examples are presented to demonstrate its capabilities

the em algorithm for kernel matrix completion with auxiliary data in biological data it is often the case that observed data are available only for a subset of samples when a kernel matrix is derived from such data we have to leave the entries for unavailable samples as missing in this paper the missing entries are completed by exploiting an auxiliary kernel matrix derived from another information source the parametric model of kernel matrices is created as a set of spectral variants of the auxiliary kernel matrix and the missing entries are estimated by fitting this model to the existing entries for model fitting we adopt the em algorithm distinguished from the em algorithm of dempster et al 1977 based on the information geometry of positive definite matrices we will report promising results on bacteria clustering experiments using two marker sequences 16s and gyrb

multiview hessian regularization for image annotation the rapid development of computer hardware and internet technology makes large scale data dependent models computationally tractable and opens a bright avenue for annotating images through innovative machine learning algorithms semisupervised learning ssl therefore received intensive attention in recent years and was successfully deployed in image annotation one representative work in ssl is laplacian regularization lr which smoothes the conditional distribution for classification along the manifold encoded in the graph laplacian however it is observed that lr biases the classification function toward a constant function that possibly results in poor generalization in addition lr is developed to handle uniformly distributed data or single view data although instances or objects such as images and videos are usually represented by multiview features such as color shape and texture in this paper we present multiview hessian regularization mhr to address the above two problems in lr based image annotation in particular mhr optimally combines multiple hr each of which is obtained from a particular view of instances and steers the classification function that varies linearly along the data manifold we apply mhr to kernel least squares and support vector machines as two examples for image annotation extensive experiments on the pascal voc 07 dataset validate the effectiveness of mhr by comparing it with baseline algorithms including lr and hr

a maximum likelihood approach to density estimation with semidefinite programming density estimation plays an important and fundamental role in pattern recognition machine learning and statistics in this article we develop a parametric approach to univariate or low dimensional density estimation based on semidefinite programming sdp our density model is expressed as the product of a nonnegative polynomial and a base density such as normal distribution exponential distribution and uniform distribution when the base density is specified the maximum likelihood estimation of the polynomial is formulated as a variant of sdp that is solved in polynomial timewith the interior point methods since the base density typically contains just one or two parameters computation of the maximum likelihood estimate reduces to a one or two dimensional easy optimization problem with this use of sdp thus the rigorous maximum likelihood estimate can be computed in our approach furthermore such conditions as symmetry and unimodality of the density function can be easily handled within this framework aic is used to choose the best model through applications to several instances we demonstrate flexibility of the model and performance of the proposed procedure combination with amixture approach is also presented the proposed approach has possible other applications beyond density estimation this point is clarified through an application to the maximum likelihood estimation of the intensity function of a nonstationary poisson process

contextual object localization with multiple kernel nearest neighbor recently many object localization models have shown that incorporating contextual cues can greatly improve accuracy over using appearance features alone therefore many of these models have explored different types of contextual sources but only considering one level of contextual interaction at the time thus what context could truly contribute to object localization through integrating cues from all levels simultaneously remains an open question moreover the relative importance of the different contextual levels and appearance features across different object classes remains to be explored here we introduce a novel framework for multiple class object localization that incorporates different levels of contextual interactions we study contextual interactions at the pixel region and object level based upon three different sources of context semantic boundary support and contextual neighborhoods our framework learns a single similarity metric from multiple kernels combining pixel and region interactions with appearance features and then applies a conditional random field to incorporate object level interactions to effectively integrate different types of feature descriptions we extend the large margin nearest neighbor to a novel algorithm that supports multiple kernels we perform experiments on three challenging image databases graz 02 msrc and pascal voc 2007 experimental results show that our model outperforms current state of the art contextual frameworks and reveals individual contributions for each contextual interaction level as well as appearance features indicating their relative importance for object localization

on the convergence rate of l p norm multiple kernel learning we derive an upper bound on the local rademacher complexity of lp norm multiple kernel learning which yields a tighter excess risk bound than global approaches previous local approaches analyzed the case p 1 only while our analysis covers all cases 1 p assuming the different feature mappings corresponding to the different kernels to be uncorrelated we also show a lower bound that shows that the bound is tight and derive consequences regarding excess loss namely fast convergence rates of the order o n 1 a where is the minimum eigenvalue decay rate of the individual kernels

multik mhks a novel multiple kernel learning algorithm in this paper we develop a new effective multiple kernel learning algorithm first we map the input data into m different feature spaces by m empirical kernels where each generated feature space is taken as one view of the input space then through borrowing the motivating argument from canonical correlation analysis cca that can maximally correlate the m views in the transformed coordinates we introduce a special term called inter function similarity loss rifsi into the existing regularization framework so as to guarantee the agreement of multiview outputs in implementation we select the modification of ho kashyap algorithm with squared approximation of the misclassification errors mhks as the incorporated paradigm and the experimental results on benchmark data sets demonstrate the feasibility and effectiveness of the proposed algorithm named multik mhks

level set based deformation methods for adaptive grids a new method for generating adaptive moving grids is formulated based on physical quantities level set functions are used to construct the adaptive grids which are solutions of the standard level set evolution equation with the cartesian coordinates as initial values the intersection points of the level sets of the evolving functions form a new grid at each time the velocity vector in the evolution equation is chosen according to a monitor function and is equal to the node velocity a uniform grid is then deformed to a moving grid with desired cell volume distribution at each time the method achieves precise control over the jacobian determinant of the grid mapping as the traditional deformation method does the new method is consistent with the level set approach to dynamic moving interface problems

sublinear time algorithms for approximate semidefinite programming we consider semidefinite optimization in a saddle point formulation where the primal solution is in the spectrahedron and the dual solution is a distribution over affine functions we present an approximation algorithm for this problem that runs in sublinear time in the size of the data to the best of our knowledge this is the first algorithm to achieve this our algorithm is also guaranteed to produce low rank solutions we further prove lower bounds on the running time of any algorithm for this problem showing that certain terms in the running time of our algorithm cannot be further improved finally we consider a non affine version of the saddle point problem and give an algorithm that under certain assumptions runs in sublinear time

pac bayesian generalisation error bounds for gaussian process classification approximate bayesian gaussian process gp classification techniques are powerful non parametric learning methods similar in appearance and performance to support vector machines based on simple probabilistic models they render interpretable results and can be embedded in bayesian frameworks for model selection feature selection etc in this paper by applying the pac bayesian theorem of mcallester 1999a we prove distribution free generalisation error bounds for a wide range of approximate bayesian gp classification techniques we also provide a new and much simplified proof for this powerful theorem making use of the concept of convex duality which is a backbone of many machine learning techniques we instantiate and test our bounds for two particular gpc techniques including a recent sparse method which circumvents the unfavourable scaling of standard gp algorithms as is shown in experiments on a real world task the bounds can be very tight for moderate training sample sizes to the best of our knowledge these results provide the tightest known distribution free error bounds for approximate bayesian gpc methods giving a strong learning theoretical justification for the use of these techniques

discriminative semi supervised feature selection via manifold regularization feature selection has attracted a huge amount of interest in both research and application communities of data mining we consider the problem of semi supervised feature selection where we are given a small amount of labeled examples and a large amount of unlabeled examples since a small number of labeled samples are usually insufficient for identifying the relevant features the critical problem arising from semi supervised feature selection is how to take advantage of the information underneath the unlabeled data to address this problem we propose a novel discriminative semi supervised feature selection method based on the idea of manifold regularization the proposed approach selects features through maximizing the classification margin between different classes and simultaneously exploiting the geometry of the probability distribution that generates both labeled and unlabeled data in comparison with previous semi supervised feature selection algorithms our proposed semi supervised feature selection method is an embedded feature selection method and is able to find more discriminative features we formulate the proposed feature selection method into a convex concave optimization problem where the saddle point corresponds to the optimal solution to find the optimal solution the level method a fairly recent optimization method is employed we also present a theoretic proof of the convergence rate for the application of the level method to our problem empirical evaluation on several benchmark data sets demonstrates the effectiveness of the proposed semi supervised feature selection method

a geometric discretization and a simple implementation for variational mesh generation and adaptation we present a simple direct discretization for functionals used in the variational mesh generation and adaptation meshing functionals are discretized on simplicial meshes and the jacobian matrix of the continuous coordinate transformation is approximated by the jacobian matrices of affine mappings between elements the advantage of this direct geometric discretization is that it preserves the basic geometric structure of the continuous functional which is useful in preventing strong decoupling or loss of integral constraints satisfied by the functional moreover the discretized functional is a function of the coordinates of mesh vertices and its derivatives have a simple analytical form which allows a simple implementation of variational mesh generation and adaptation on computer since the variational mesh adaptation is the base for a number of adaptive moving mesh and mesh smoothing methods the result in this work can be used to develop simple implementations of those methods numerical examples are given

stochastic approximation method with gradient averaging for unconstrained problems a stochastic approximation algorithm is studied in which random gradient estimates are averaged by an auxiliary filter convergence of the algorithm is proved under a rather general noise condition a practical version of the method is described and numerical results are given

adaptive zoning for singular problems in two dimensions abstract winslow s method for the automatic generation of computation meshes is extended to adaptively vary the zone sizes and orthogonality of grid lines in the resulting mesh through simple analysis and numerical examples the adaptive mesh is shown to give significant increases in accuracy in the computation of singular problems

multiset canonical correlations using globality preserving projections with applications to feature extraction and recognition multiset features extracted from the same patterns always represent different characteristics of data thus it is very valuable to perform the extraction on multiple feature sets this paper addresses the issue of multiset correlation feature extraction mcfe in multiple feature representations a novel method is proposed to carry out the mcfe for classification called multiset canonical correlations using globality preserving projections mcc gps which can perform joint dimensionality reduction for high dimensional data mcc gp integrates correlational characteristics of feature pairs and global geometric information of data in the transformed low dimensional space this makes mcc gps have better discriminant ability than a previous method proposed by the authors called multiset integrated canonical correlation analysis micca which only considers correlations for recognition tasks furthermore mcc gp can subsume two popular feature extraction methods into its framework under some constraints this also provides a new insight for these two methods the proposed method is applied to pattern recognition and examined using the coil 100 and eth 80 object databases and ar cmu pie and yale face databases extensive experimental results show that mcc gp outperforms micca and multiset canonical correlation analysis in terms of classification accuracy and efficiency

learning the kernel matrix by maximizing a kfd based class separability criterion the advantage of a kernel method often depends critically on a proper choice of the kernel function a promising approach is to learn the kernel from data automatically in this paper we propose a novel method for learning the kernel matrix based on maximizing a class separability criterion that is similar to those used by linear discriminant analysis lda and kernel fisher discriminant kfd it is interesting to note that optimizing this criterion function does not require inverting the possibly singular within class scatter matrix which is a computational problem encountered by many lda and kfd methods we have conducted experiments on both synthetic data and real world data from uci and feret showing that our method consistently outperforms some previous kernel learning methods

mixing linear svms for nonlinear classification in this paper we address the problem of combining linear support vector machines svms for classification of large scale nonlinear datasets the motivation is to exploit both the efficiency of linear svms lsvms in learning and prediction and the power of nonlinear svms in classification to this end we develop a lsvm mixture model that exploits a divide and conquer strategy by partitioning the feature space into subregions of linearly separable datapoints and learning a lsvm for each of these regions we do this implicitly by deriving a generative model over the joint data and label distributions consequently we can impose priors on the mixing coefficients and do implicit model selection in a top down manner during the parameter estimation process this guarantees the sparsity of the learned model experimental results show that the proposed method can achieve the efficiency of lsvms in the prediction phase while still providing a classification performance comparable to nonlinear svms

a robust elliptic grid generator a variational principle is proposed that results in a robust elliptic grid generator having many of the strengths of original winslow or homogeneous thompson thames mastin method httm the new grid generator places grid lines more uniformly over the domain than does httm without loss of orthogonality numerically generated examples are given to demonstrate these effects grid quality measures are introduced to quantify differences between discrete grids both the httm and the new grid generator can generate folded grids on certain pathological regions but overall they are very robust grid weighting for solution adaptive calculations is briefly considered generalization of the new method to surface and volume grid generation is straightforward

generalized hierarchical kernel learning this paper generalizes the framework of hierarchical kernel learning hkl and illustrates its utility in the domain of rule learning hkl involves multiple kernel learning over a set of given base kernels assumed to be embedded on a directed acyclic graph this paper proposes a two fold generalization of hkl the first is employing a generic l1 lp block norm regularizer 1 2 that alleviates a key limitation of the hkl formulation the second is a generalization to the case of multi class multi label and more generally multi task applications the main technical contribution of this work is the derivation of a highly specialized partial dual of the proposed generalized hkl formulation and an efficient mirror descent based active set algorithm for solving it importantly the generic regularizer enables the proposed formulation to be employed in the rule ensemble learning rel where the goal is to construct an ensemble of conjunctive propositional rules experiments on benchmark rel data sets illustrate the efficacy of the proposed generalizations

building sparse multiple kernel svm classifiers the support vector machines svms have been very successful in many machine learning problems however they can be slow during testing because of the possibly large number of support vectors obtained recently wu 2005 proposed a sparse formulation that restricts the svm to use a small number of expansion vectors in this paper we further extend this idea by integrating with techniques from multiple kernel learning mkl the kernel function in this sparse svm formulation no longer needs to be fixed but can be automatically learned as a linear combination of kernels two formulations of such sparse multiple kernel classifiers are proposed the first one is based on a convex combination of the given base kernels while the second one uses a convex combination of the so called ldquoequivalentrdquo kernels empirically the second formulation is particularly competitive experiments on a large number of toy and real world data sets show that the resultant classifier is compact and accurate and can also be easily trained by simply alternating linear program and standard svm solver

ensemble estimators for multivariate entropy estimation the problem of estimation of density functionals like entropy and mutual information has received much attention in the statistics and information theory communities a large class of estimators of functionals of the probability density suffer from the curse of dimensionality wherein the mean squared error decays increasingly slowly as a function of the sample size t as the dimension d of the samples increases in particular the rate is often glacially slow of order o t d where 0 is a rate parameter examples of such estimators include kernel density estimators k nearest neighbor k nn density estimators k nn entropy estimators intrinsic dimension estimators and other examples in this paper we propose a weighted affine combination of an ensemble of such estimators where optimal weights can be chosen such that the weighted estimator converges at a much faster dimension invariant rate of o t1 furthermore we show that these optimal weights can be determined by solving a convex optimization problem which can be performed offline and does not require training data we illustrate the superior performance of our weighted estimator for two important applications 1 estimating the panter dite distortion rate factor and 2 estimating the shannon entropy for testing the probability distribution of a random sample

empirical margin distributions and bounding the generalization error of combined classifiers we prove new probabilistic upper bounds on generalization error of complex classifiers that are combinations of simple classifiers such combinations could be implemented by neural networks or by voting methods of combining the classifiers such as boosting and bagging the bounds are in terms of the empirical distribution of the margin of the combined classifier they are based on the methods of the theory of gaussian and empirical processes comparison inequalities symmetrization method concentration inequalities and they improve previous results of bartlett 1998 on bounding the generalization error of neural networks in terms of ell 1 norms of the weights of neurons and of schapire freund bartlett and lee 1998 on bounding the generalization error of boosting we also obtain rates of convergence in levy distance of empirical margin distribution to the true margin distribution uniformly over the classes of classifiers and prove the optimality of these rates

interactive method of variational grid generation abstract in the paper the elliptic variational grid generation technique based on methods of optimal control for distributed systems is presented the interactive nature of the method consists in adaptive changing of the golas of correction steps in the process of grid generation the corrections concern orthogonality and uniformity of the mesh

localized multiple kernel learning with dynamical clustering and matrix regularization localized multiple kernel learning lmkl is an attractive strategy for combining multiple heterogeneous features with regard to their discriminative power for each individual sample however the learning of numerous local solutions may not scale well even for a moderately sized training set and the independently learned local models may suffer from overfitting hence in existing local methods the distributed samples are typically assumed to share the same weights and various unsupervised clustering methods are applied as preprocessing in this paper to enable the learner to discover and benefit from the underlying local coherence and diversity of the samples we incorporate the clustering procedure into the canonical support vector machine based lmkl framework then to explore the relatedness among different samples which has been ignored in a vector ell p norm analysis we organize the cluster specific kernel weights into a matrix and introduce a matrix based extension of the ell p norm for constraint enforcement by casting the joint optimization problem as a problem of alternating optimization we show how the cluster structure is gradually revealed and how the matrix regularized kernel weights are obtained a theoretical analysis of such a regularizer is performed using a rademacher complexity bound and complementary empirical experiments on real world data sets demonstrate the effectiveness of our technique

a family of simple non parametric kernel learning algorithms previous studies of non parametric kernel learning npkl usually formulate the learning task as a semi definite programming sdp problem that is often solved by some general purpose sdp solvers however for n data examples the time complexity of npkl using a standard interior point sdp solver could be as high as o n6 5 which prohibits npkl methods applicable to real applications even for data sets of moderate size in this paper we present a family of efficient npkl algorithms termed simplenpkl which can learn non parametric kernels from a large set of pairwise constraints efficiently in particular we propose two efficient simplenpkl algorithms one is simplenpkl algorithm with linear loss which enjoys a closed form solution that can be efficiently computed by the lanczos sparse eigen decomposition technique another one is simplenpkl algorithm with other loss functions including square hinge loss hinge loss square loss that can be re formulated as a saddle point optimization problem which can be further resolved by a fast iterative algorithm in contrast to the previous npkl approaches our empirical results show that the proposed new technique maintaining the same accuracy is significantly more efficient and scalable finally we also demonstrate that the proposed new technique is also applicable to speed up many kernel learning tasks including colored maximum variance unfolding minimum volume embedding and structure preserving embedding

symmetric rbf classifier for nonlinear detection in multiple antenna aided systems in this paper we propose a powerful symmetric radial basis function rbf classifier for nonlinear detection in the so called overloaded multiple antenna aided communication systems by exploiting the inherent symmetry property of the optimal bayesian detector the proposed symmetric rbf classifier is capable of approaching the optimal classification performance using noisy training data the classifier construction process is robust to the choice of the rbf width and is computationally efficient the proposed solution is capable of providing a signal to noise ratio snr gain in excess of 8 db against the powerful linear minimum bit error rate ber benchmark when supporting four users with the aid of two receive antennas or seven users with four receive antenna elements

low rank decomposition meets kernel learning a generalized nystrom method abstract low rank matrix decomposition and kernel learning are two useful techniques in building advanced learning systems low rank decomposition can greatly reduce the computational cost of manipulating large kernel matrices however existing approaches are mostly unsupervised and do not incorporate side information such as class labels making the decomposition less effective for a specific learning task on the other hand kernel learning techniques aim at constructing kernel matrices whose structure is well aligned with the learning target which improves the generalization performance of kernel methods however most kernel learning approaches are computationally very expensive to obtain the advantages of both techniques and address their limitations in this paper we propose a novel kernel low rank decomposition formulation called the generalized nystrom method our approach inherits the linear time and space complexity via matrix decomposition while at the same time fully exploits partial label information in computing task dependent decomposition in addition the resultant low rank factors can generalize to arbitrary new samples rendering great flexibility in inductive learning scenarios we further extend the algorithm to a multiple kernel learning setup the experimental results on semi supervised classification demonstrate the usefulness of the proposed method

mathematical aspects of variational grid generation ii abstract variational grid generation techniques are now used to produce grids suitable for solving numerical partial differential equations in irregular geometries in this paper the existence and uniqueness of solutions of the volume and smoothness problems that are used in variational grid generation are studied an analysis of the euler lagrange el equations near the identity shows that the volume problem is difficult these variational problems use a reference grid to specify the properties of the desired grid replication of reference grid properties is analyzed examples are given that show the effectiveness of the reference grid concept

area control in generating smooth and convex grids over general plane regions a new method adaptive smoothness functional to produce convex and smooth grids over general plane regions has been introduced in a recent work by the authors 10 this method belongs to the variational grid generation approach theoretical results showing that a convex grid over a region is obtained when this method is applied were presented the basic assumption was that at least one convex grid exists a procedure to control large cells bilateral smoothness functional in addition to smoothness and convexity was also presented experimental results showing the effectiveness of these methods were reported however no theoretical results were reported assuring that the area control can be always exerted this article continues the same line of research introducing a new version of the bilateral smoothness functional that improves the control of large areas unlike the former method theoretical results show the effectiveness of the new bilateral smoothness functional to exert such control optimal grids obtained with the new functional are compared with those reached using the older version demostrating the improvement

a kernel optimization method based on the localized kernel fisher criterion it is widely recognized that whether the selected kernel matches the data determines the performance of kernel based methods ideally it is expected that the data is linearly separable in the kernel induced feature space therefore fisher linear discriminant criterion can be used as a cost function to optimize the kernel function however the data may not be linearly separable even after kernel transformation in many applications e g the data may exist as multimodally distributed structure in this case a nonlinear classifier is preferred and obviously fisher criterion is not a suitable choice as kernel optimization rule motivated by this issue we propose a localized kernel fisher criterion instead of traditional fisher criterion as the kernel optimization rule to increase the local margins between embedded classes in kernel induced feature space experimental results based on some benchmark data and measured radar high resolution range profile hrrp data show that the classification performance can be improved by using the proposed method

feature selection and kernel learning for local learning based clustering the performance of the most clustering algorithms highly relies on the representation of data in the input space or the hilbert space of kernel methods this paper is to obtain an appropriate data representation through feature selection or kernel learning within the framework of the local learning based clustering llc wu and scholkopf 2006 method which can outperform the global learning based ones when dealing with the high dimensional data lying on manifold specifically we associate a weight to each feature or kernel and incorporate it into the built in regularization of the llc algorithm to take into account the relevance of each feature or kernel for the clustering accordingly the weights are estimated iteratively in the clustering process we show that the resulting weighted regularization with an additional constraint on the weights is equivalent to a known sparse promoting penalty hence the weights of those irrelevant features or kernels can be shrunk toward zero extensive experiments show the efficacy of the proposed methods on the benchmark data sets

selective multiple kernel learning for classification with ensemble strategy abstract multiple kernel learning mkl aims to seek a better result than single kernel learning by combining a compact set of sub kernels however mkl with l1 norm easily discards the sub kernels with complementary information and mkl with lp norm p 2 often gets the redundant solution to address these problems a selective multiple kernel learning smkl method inspired by ensemble learning el is proposed comparing mkl with lp norm p 2 smkl obtains a sparse solution by a pre selection procedure comparing mkl with l1 norm smkl preserves the sub kernels with complementary information by guaranteeing the high discrimination and large diversity of pre selected sub kernels for quantifying the discrimination and diversity of sub kernels a new kernel evaluation is designed smkl reduces the scale of mkl optimization and saves the memory storing of the sub kernels which extends the scale of problem that mkl could solve specially a fast smkl method using l norm constraint is focused which needs no mkl optimization process it means that the memory is hardly a limitation for mkl with the large scale problem experiments state that our method is effective for classification

large margin multi viewinformation bottleneck in this paper we extend the theory of the information bottleneck ib to learning from examples represented by multi view features we formulate the problem as one of encoding a communication system with multiple senders each of which represents one view of the data based on the precise components filtered out from multiple information sources through a bottleneck a margin maximization approach is then used to strengthen the discrimination of the encoder by improving the code distance within the frame of coding theory the resulting algorithm therefore inherits all the merits of the ib principle and coding theory it has two distinct advantages over existing algorithms namely that our method finds a tradeoff between the accuracy and complexity of the multi view model and that the encoded multi view data retains sufficient discrimination for classification we also derive the robustness and generalization error bound of the proposed algorithm and reveal the specific properties of multi view learning first the complementarity of multi view features guarantees the robustness of the algorithm second the consensus of multi view features reduces the empirical rademacher complexity of the objective function enhances the accuracy of the solution and improves the generalization error bound of the algorithm the resulting objective function is solved efficiently using the alternating direction method experimental results on annotation classification and recognition tasks demonstrate that the proposed algorithm is promising for practical applications

learning the kernel function via regularization we study the problem of finding an optimal kernel from a prescribed convex set of kernels k for learning a real valued function by regularization we establish for a wide variety of regularization functionals that this leads to a convex optimization problem and for square loss regularization we characterize the solution of this problem we show that although k may be an uncountable set the optimal kernel is always obtained as a convex combination of at most m 2 basic kernels where m is the number of data examples in particular our results apply to learning the optimal radial kernel or the optimal dot product kernel

multi kernel learning with online batch optimization in recent years there has been a lot of interest in designing principled classification algorithms over multiple cues based on the intuitive notion that using more features should lead to better performance in the domain of kernel methods a principled way to use multiple features is the multi kernel learning mkl approach here we present a mkl optimization algorithm based on stochastic gradient descent that has a guaranteed convergence rate we directly solve the mkl problem in the primal formulation by having a p norm formulation of mkl we introduce a parameter that controls the level of sparsity of the solution while leading to an easier optimization problem we prove theoretically and experimentally that 1 our algorithm has a faster convergence rate as the number of kernels grows 2 the training complexity is linear in the number of training examples 3 very few iterations are sufficient to reach good solutions experiments on standard benchmark databases support our claims

learning the kernel matrix with semidefinite programming kernel based learning algorithms work by embedding the data into a euclidean space and then searching for linear relations among the embedded data points the embedding is performed implicitly by specifying the inner products between each pair of points in the embedding space this information is contained in the so called kernel matrix a symmetric and positive semidefinite matrix that encodes the relative positions of all points specifying this matrix amounts to specifying the geometry of the embedding space and inducing a notion of similarity in the input space classical model selection problems in machine learning in this paper we show how the kernel matrix can be learned from data via semidefinite programming sdp techniques when applied to a kernel matrix associated with both training and test data this gives a powerful transductive algorithm using the labeled part of the data one can learn an embedding also for the unlabeled part the similarity between test points is inferred from training points and their labels importantly these learning problems are convex so we obtain a method for learning both the model class and the function without local minima furthermore this approach leads directly to a convex method for learning the 2 norm soft margin parameter in support vector machines solving an important open problem

a practical guide to direct optimization for planar grid generation abstract in this paper we provide a guide to using the direct optimization formulation of variational grid generation particular emphasis is placed on the smoothness or length functional this is undoubtedly the most important functional in variational grid generation producing smooth grids and ensuring well posedness of the minimization problem when combined with more unruly functionals unfortunately in its most primary form length can produce folded grids when used with nonconvex geometries historically there have been two solutions to this dilemma using an inverse mapping the famous winslow generator or augmenting the functional with others that promote unicity both strategies have the drawback that the resulting minimization problem becomes complicated and expensive as another alternative we introduce a generalized strategy for length which does not use inverse mappings or auxiliary functionals but makes strong use of reference grids this strategy provides flexibility in controlling grid quality and its minimization problems can be solved using a simple multigrid algorithm yielding a robust grid generation scheme with optimal complexity we also survey recent developments in the use of variational grid generation in the form of direct optimization for the alignment problem

feature fusion using locally linear embedding for classification in most complex classification problems many types of features have been captured or extracted feature fusion is used to combine features for better classification and to reduce data dimensionality kernel based feature fusion methods are very effective for classification but they do not reduce data dimensionality in this brief we propose an effective feature fusion method using locally linear embedding lle the proposed method overcomes the limitations of lle which could not handle different types of features and is inefficient for classification we propose an efficient algorithm to solve the optimization problem in obtaining weights of different features and design an efficient method for lle based classification in comparison to other kernel based feature fusion methods the proposed method fuses features to a significantly lower dimensional feature space with the same discriminant power we have conducted experiments to demonstrate the effectiveness of the proposed feature fusion method

soft margin multiple kernel learning multiple kernel learning mkl has been proposed for kernel methods by learning the optimal kernel from a set of predefined base kernels however the traditional l 1 rm mkl method often achieves worse results than the simplest method using the average of base kernels i e average kernel in some practical applications in order to improve the effectiveness of mkl this paper presents a novel soft margin perspective for mkl specifically we introduce an additional slack variable called kernel slack variable to each quadratic constraint of mkl which corresponds to one support vector machine model using a single base kernel we first show that l 1 rm mkl can be deemed as hard margin mkl and then we propose a novel soft margin framework for mkl three commonly used loss functions including the hinge loss the square hinge loss and the square loss can be readily incorporated into this framework leading to the new soft margin mkl objective functions many existing mkl methods can be shown as special cases under our soft margin framework for example the hinge loss soft margin mkl leads to a new box constraint for kernel combination coefficients using different hyper parameter values for this formulation we can inherently bridge the method using average kernel l 1 rm mkl and the hinge loss soft margin mkl the square hinge loss soft margin mkl unifies the family of elastic net constraint regularizer based approaches and the square loss soft margin mkl incorporates l 2 rm mkl naturally moreover we also develop efficient algorithms for solving both the hinge loss and square hinge loss soft margin mkl comprehensive experimental studies for various mkl algorithms on several benchmark data sets and two real world applications including video action recognition and event recognition demonstrate that our proposed algorithms can efficiently achieve an effective yet sparse solution for mkl

variational curve and surface grid generation abstract variational algorithms that control the lengths of grid lines cell areas and the orthogonality of grid lines can be used for generating boundary conforming grids on surfaces additional geometric control is provided by using a reference grid while solution adaptivity is achieved by using weights in a typical application the reference grid can be used to produce an exponential compression of the grid at a boundary while the solution adaptive weights are used to make the grid spacing inversely proportional to the gradient when the gradient is large of some solution being computed on the grid the grid is adapted on both the interior and boundary of the surface the algorithm performs these tasks with exceptional precision as demonstrated in the examples presented here