learning the kernel with hyperkernels this paper addresses the problem of choosing a kernel suitable for estimation with a support vector machine hence further automating machine learning this goal is achieved by defining a reproducing kernel hilbert space on the space of kernels itself such a formulation leads to a statistical estimation problem similar to the problem of minimizing a regularized risk functional we state the equivalent representer theorem for the choice of kernels and present a semidefinite programming formulation of the resulting optimization problem several recipes for constructing hyperkernels are provided as well as the details of common machine learning problems experimental results for classification regression and novelty detection on uci data show the feasibility of our approach

optimal isoparametric finite elements and error estimates for domains involving curved boundaries we describe a practical procedure for the triangulation of arbitrary n dimensional regular bounded domains by isoparametric simplicial elements preserving the optimal order of accuracy along similar lines we prove general error estimates for the finite element solution of second order elliptic problems various kinds of nonhomogeneous boundary conditions can be dealt with such as dirichlet neumann and robin boundary conditions

on implementing a primal dual interior point method for conic quadratic optimization based on the work of the nesterov and todd on self scaled cones an implementation of a primal dual interior point method for solving large scale sparse conic quadratic optimization problems is presented the main features of the implementation are it is based on a homogeneous and self dual model it handles rotated quadratic cones directly it employs a mehrotra type predictor corrector extension and sparse linear algebra to improve the computational efficiency finally the implementation exploits fixed variables which naturally occurs in many conic quadratic optimization problems this is a novel feature for our implementation computational results are also presented to document that the implementation can solve very large problems robustly and efficiently

multi class discriminant kernel learning via convex programming regularized kernel discriminant analysis rkda performs linear discriminant analysis in the feature space via the kernel trick its performance depends on the selection of kernels in this paper we consider the problem of multiple kernel learning mkl for rkda in which the optimal kernel matrix is obtained as a linear combination of pre specified kernel matrices we show that the kernel learning problem in rkda can be formulated as convex programs first we show that this problem can be formulated as a semidefinite program sdp based on the equivalence relationship between rkda and least square problems in the binary class case we propose a convex quadratically constrained quadratic programming qcqp formulation for kernel learning in rkda a semi infinite linear programming silp formulation is derived to further improve the efficiency we extend these formulations to the multi class case based on a key result established in this paper that is the multi class rkda kernel learning problem can be decomposed into a set of binary class kernel learning problems which are constrained to share a common kernel based on this decomposition property sdp formulations are proposed for the multi class case furthermore it leads naturally to qcqp and silp formulations as the performance of rkda depends on the regularization parameter we show that this parameter can also be optimized in a joint framework with the kernel extensive experiments have been conducted and analyzed and connections to other algorithms are discussed

3d fem formulations of limit analysis methods for porous pressure sensitive materials summary the first purpose of this paper is the numerical formulation of the three general limit analysis methods for problems involving pressure sensitive materials that is the static classic and mixed kinematic methods applied to problems with drucker prager mises schleicher or green materials in each case quadratic or rotated quadratic cone programming is considered to solve the final optimization problems leading to original and efficient numerical formulations as a second purpose the resulting codes are applied to non classic 3d problems that is the gurson like hollow sphere problem with these materials as matrices to this end are first presented the 3d finite element implementations of the static and kinematic classic methods of limit analysis together with a mixed method formulated to give also a purely kinematic result discontinuous stress and velocity fields are included in the analysis the static and the two kinematic approaches are compared afterwards in the hydrostatic loading case whose exact solution is known for the three cases of matrix then the static and the mixed approaches are used to assess the available approximate criteria for porous drucker prager mises schleicher and green materials copyright 2013 john wiley sons ltd

product form cholesky factorization in interior point methods for second order cone programming second order cone programming socp problems are typically solved by interior point methods as in linear programming lp interior point methods can in theory solve socps in polynomial time and can in practice exploit sparsity in the problem data specifically when cones of large dimension are present the density that results in the normal equations that are solved at each iteration can be remedied in a manner similar to the treatment of dense columns in an lp here we propose a product form cholesky factorization pfcf approach and show that it is more numerically stable than the alternative sherman morrison woodbury approach we derive several pfcf variants and compare their theoretical perfomance finally we prove that the elements of l in the cholesky factorizations ldlt that arise in interior point methods for socp are uniformly bounded as the duality gap tends to zero as long as the iterates remain is some conic neighborhood of the cental path

optimal finite element interpolation on curved domains this paper is devoted to a general theory of approximation of functions in finite element spaces in particular the case is considered where the functions to be interpolated are on the one hand not very smooth and on the other are defined on curved domains thus a number of well known optimal interpolation results are generalized

optimizing the kernel in the empirical feature space in this paper we present a method of kernel optimization by maximizing a measure of class separability in the empirical feature space an euclidean space in which the training data are embedded in such a way that the geometrical structure of the data in the feature space is preserved employing a data dependent kernel we derive an effective kernel optimization algorithm that maximizes the class separability of the data in the empirical feature space it is shown that there exists a close relationship between the class separability measure introduced here and the alignment measure defined recently by cristianini extensive simulations are carried out which show that the optimized kernel is more adaptive to the input data and leads to a substantial sometimes significant improvement in the performance of various data classification algorithms

building sparse multiple kernel svm classifiers the support vector machines svms have been very successful in many machine learning problems however they can be slow during testing because of the possibly large number of support vectors obtained recently wu 2005 proposed a sparse formulation that restricts the svm to use a small number of expansion vectors in this paper we further extend this idea by integrating with techniques from multiple kernel learning mkl the kernel function in this sparse svm formulation no longer needs to be fixed but can be automatically learned as a linear combination of kernels two formulations of such sparse multiple kernel classifiers are proposed the first one is based on a convex combination of the given base kernels while the second one uses a convex combination of the so called ldquoequivalentrdquo kernels empirically the second formulation is particularly competitive experiments on a large number of toy and real world data sets show that the resultant classifier is compact and accurate and can also be easily trained by simply alternating linear program and standard svm solver

finite element analysis of varitional crimes for a quasilinear elliptic problem in 3d we examine a finite element approximation of a quasilinear boundary value elliptic problem in a three dimensional bounded convex domain with a smooth boundary the domain is approximated by a polyhedron and a numerical integration is taken into account we apply linear tetrahedral finite elements and prove the convergence of approximate solutions on polyhedral domains in the w 1 2 norm to the true solution without any additional regularity assumptions

discrete maximum principles for finite element solutions of nonlinear elliptic problems with mixed boundary conditions one of the most important problems in numerical simulations is the preservation of qualitative properties of solutions of the mathematical models by computed approximations for problems of elliptic type one of the basic properties is the continuous maximum principle in our work we present several variants of the maximum principles and their discrete counterparts for scalar second order nonlinear elliptic problems with mixed boundary conditions the problems considered are numerically solved by the continuous piecewise linear finite element approximations built on simplicial meshes sufficient conditions providing the validity of the corresponding discrete maximum principles are presented geometrically they mean that the employed meshes have to be of acute or nonobtuse type depending of the type of the problem finally some examples of real life problems where the preservation of maximum principles plays an important role are presented

limit analysis of plates using the efg method and second order cone programming the meshless element free galerkin efg method is extended to allow computation of the limit load of plates a kinematic formulation that involves approximating the displacement field using the moving least squares technique is developed only one displacement variable is required for each efg node ensuring that the total number of variables in the resulting optimization problem is kept to a minimum with far fewer variables being required compared with finite element formulations using compatible elements a stabilized conforming nodal integration scheme is extended to plastic plate bending problems the evaluation of integrals at nodal points using curvature smoothing stabilization both keeps the size of the optimization problem small and also results in stable and accurate solutions difficulties imposing essential boundary conditions are overcome by enforcing displacements at the nodes directly the formulation can be expressed as the problem of minimizing a sum of euclidean norms subject to a set of equality constraints this non smooth minimization problem can be transformed into a form suitable for solution using second order cone programming the procedure is applied to several benchmark beam and plate problems and is found in practice to generate good upper bound solutions for benchmark problems

standard 3 components of type sl 4 2 abstract aschbacher s recent work on finite simple groups g with e g 3 has led to the following problem find all finite simple groups g which contain an element b of order 3 such that c b contains a normal quasi simple subgroup l with l z l sl 4 2 and such that certain further conditions hold this paper solves that problem

the finite element method for nonlinear elliptic equations with discontinuous coefficients the study of the finite element approximation to nonlinear second order elliptic boundary value problems with discontinuous coefficients is presented in the case of mixed dirichlet neumann boundary conditions the change in domain and numerical integration are taken into account with the assumptions which guarantee that the corresponding operator is strongly monotone and lipschitz continuous the following convergence results are proved 1 the rate of convergenceo h if the exact solutionu h 1 is piecewise of classh 1 0 1 2 the convergence without any rate of convergence ifu h 1 only

nonnegative matrix factorization in polynomial feature space plenty of methods have been proposed in order to discover latent variables features in data sets such approaches include the principal component analysis pca independent component analysis ica factor analysis fa etc to mention only a few a recently investigated approach to decompose a data set with a given dimensionality into a lower dimensional space is the so called nonnegative matrix factorization nmf its only requirement is that both decomposition factors are nonnegative to approximate the original data the minimization of the nmf objective function is performed in the euclidean space where the difference between the original data and the factors can be minimized by employing l 2 norm in this paper we propose a generalization of the nmf algorithm by translating the objective function into a hilbert space also called feature space under nonnegativity constraints with the help of kernel functions we developed an approach that allows high order dependencies between the basis images while keeping the nonnegativity constraints on both basis images and coefficients two practical applications namely facial expression and face recognition show the potential of the proposed approach

lagrangian support vector machines an implicit lagrangian for the dual of a simple reformulation of the standard quadratic program of a linear support vector machine is proposed this leads to the minimization of an unconstrained differentiable convex function in a space of dimensionality equal to the number of classified points this problem is solvable by an extremely simple linearly convergent lagrangian support vector machine lsvm algorithm lsvm requires the inversion at the outset of a single matrix of the order of the much smaller dimensionality of the original input space plus one the full algorithm is given in this paper in 11 lines of matlab code without any special optimization tools such as linear or quadratic programming solvers this lsvm code can be used as is to solve classification problems with millions of points for example 2 million points in 10 dimensional input space were classified by a linear surface in 82 minutes on a pentium iii 500 mhz notebook with 384 megabytes of memory and additional swap space and in 7 minutes on a 250 mhz ultrasparc ii processor with 2 gigabytes of memory other standard classification test problems were also solved nonlinear kernel classification can also be solved by lsvm although it does not scale up to very large problems it can handle any positive semidefinite kernel and is guaranteed to converge a short matlab code is also given for nonlinear kernels and tested on a number of problems

system identification via sparse multiple kernel based regularization using sequential convex optimization techniques model estimation and structure detection with short data records are two issues that receive increasing interests in system identification in this paper a multiple kernel based regularization method is proposed to handle those issues multiple kernels are conic combinations of fixed kernels suitable for impulse response estimation and equip the kernel based regularization method with three features first multiple kernels can better capture complicated dynamics than single kernels second the estimation of their weights by maximizing the marginal likelihood favors sparse optimal weights which enables this method to tackle various structure detection problems e g the sparse dynamic network identification and the segmentation of linear systems third the marginal likelihood maximization problem is a difference of convex programming problem it is thus possible to find a locally optimal solution efficiently by using a majorization minimization algorithm and an interior point method where the cost of a single interior point iteration grows linearly in the number of fixed kernels monte carlo simulations show that the locally optimal solutions lead to good performance for randomly generated starting points

polynomial convergence of primal dual algorithms for the second order cone program based on the mz family of directions in this paper we study primal dual path following algorithms for the second order cone programming socp based on a family of directions that is a natural extension of the monteiro zhang mz family for semidefinite programming we show that the polynomial iteration complexity bounds of two well known algorithms for linear programming namely the short step path following algorithm of kojima et al and monteiro and adler and the predictor corrector algorithm of mizuno et al carry over to the context of socp that is they have an o loge 1 iteration complexity to reduce the duality gap by a factor of e where n is the number of second order cones since the mz type family studied in this paper includes an analogue of the alizadeh haeberly and overton pure newton direction we establish for the first time the polynomial convergence of primal dual algorithms for socp based on this search direction

barrier functions in interior point methods we show that the universal barrier function of a convex cone introduced by nesterov and nemirovskii is the logarithm of the characteristic function of the cone this interpretation demonstrates the invariance of the universal barrier under the automorphism group of the underlying cone this provides a simple method for calculating the universal barrier for homogeneous cones we identify some known barriers as the universal barrier scaled by an appropriate constant we also calculate some new universal barrier functions our results connect the field of interior point methods to several branches of mathematics such as lie groups jordan algebras siegel domains differential geometry complex analysis of several variables etc

applications of second order cone programming in a second order cone program socp a linear function is minimized over the intersection of an affine set and the product of second order quadratic cones socps are nonlinear convex problems that include linear and convex quadratic programs as special cases but are less general than semidefinite programs sdps several efficient primaldual interior point methods for socp have been developed in the last few years after reviewing the basic theory of socps we describe general families of problems that tan be recast as socps these include robust linear programming and robust leastsquares problems problems involving sums or maxima of norms or with convex hyperbolic constraints we discuss a variety of engineering applications such as filter design antenna array weight design truss design and grasping forte optimization in robotics we describe an efficient primaldual interior point method for solving socps which shares many of the features of primaldual interior point methods for linear program

presolving in linear programming most modern linear programming solvers analyze the lp problem before submitting it to optimization some examples are the solvers whizard tomlin and welch 1983 ob1 lustig et al 1994 osl forrest and tomlin 1992 sciconic 1990 and cplex bixby 1994 the purpose of the presolve phase is to reduce the problem size and to discover whether the problem is unbounded or infeasible in this paper we present a comprehensive survey of presolve methods moreover we discuss the restoration procedure in detail i e the procedure that undoes the presolve computational results on the netlib problems gay 1985 are reported to illustrate the efficiency of the presolve methods

computing limit loads by minimizing a sum of norms this paper treats the problem of computing the collapse state in limit analysis for a solid with a quadratic yield condition such as for example the von mises condition after discretization with the finite element method using divergence free elements for the plastic flow the kinematic formulation reduces to the problem of minimizing a sum of euclidean vector norms subject to a single linear constraint this is a nonsmooth minimization problem since many of the norms in the sum may vanish at the optimal point recently an efficient solution algorithm has been developed for this particular convex optimization problem in large sparse form the approach is applied to test problems in limit analysis in two different plane models plane strain and plates in the first case more than 80 of the terms in the objective function are zero in the optimal solution causing extreme ill conditioning in the second case all terms are nonzero in both cases the method works very well and problems are solved which are larger by at least an order of magnitude than previously reported the relative accuracy for the solution of the discrete problems measured by duality gap and feasibility is typically of the order 10 8

finite element solution of nonlinear elliptic problems the study of the finite element approximation to nonlinear second order elliptic boundary value problems with mixed dirichlet neumann boundary conditions is presented in the discretization variational crimes are commited approximation of the given domain by a polygonal one numerical integration with the assumption that the corresponding operator is strongly monotone and lipschitz continuous and that the exact solutionu h1 the convergence of the method is proved under the additional assumptionu h2 the rate of convergenceo h is derived without the use of green s theorem

metric and kernel learning using a linear transformation metric and kernel learning arise in several machine learning applications however most existing metric learning algorithms are limited to learning metrics over low dimensional data while existing kernel learning algorithms are often limited to the transductive setting and do not generalize to new data points in this paper we study the connections between metric learning and kernel learning that arise when studying metric learning as a linear transformation learning problem in particular we propose a general optimization framework for learning metrics via linear transformations and analyze in detail a special case of our framework that of minimizing the logdet divergence subject to linear constraints we then propose a general regularized framework for learning a kernel matrix and show it to be equivalent to our metric learning framework our theoretical connections between metric and kernel learning have two main consequences 1 the learned kernel matrix parameterizes a linear transformation kernel function and can be applied inductively to new data points 2 our result yields a constructive method for kernelizing most existing mahalanobis metric learning formulations we demonstrate our learning approach by applying it to large scale real world problems in computer vision text mining and semi supervised kernel dimensionality reduction

new support vector algorithms we propose a new class of support vector algorithms for regression and classification in these algorithms a parameter lets one effectively control the number of support vectors while this can be useful in its own right the parameterization has the additional benefit of enabling us to eliminate one of the other free parameters of the algorithm the accuracy parameter epsilon in the regression case and the regularization constant c in the classification case we describe the algorithms give some theoretical results concerning the meaning and the choice of and report experimental results

second order cone programming second order cone programming socp problems are convex optimization problems in which a linear function is minimized over the intersection of an affine linear manifold with the cartesian product of second order lorentz cones linear programs convex quadratic programs and quadratically constrained convex quadratic programs can all be formulated as socp problems as can many other problems that do not fall into these three categories these latter problems model applications from a broad range of fields from engineering control and finance to robust optimization and combinatorial optimization on the other hand semidefinite programming sdp that is the optimization problem over the intersection of an affine set and the cone of positive semidefinite matrices includes socp as a special case therefore socp falls between linear lp and quadratic qp programming and sdp like lp qp and sdp problems socp problems can be solved in polynomial time by interior point methods the computational effort per iteration required by these methods to solve socp problems is greater than that required to solve lp and qp problems but less than that required to solve sdp s of similar size and structure because the set of feasible solutions for an socp problem is not polyhedral as it is for lp and qp problems it is not readily apparent how to develop a simplex or simplex like method for socp while socp problems can be solved as sdp problems doing so is not advisable both on numerical grounds and computational complexity concerns for instance many of the problems presented in the survey paper of vandenberghe and boyd vb96 as examples of sdps can in fact be formulated as socps and should be solved as such in 2 3 below we give socp formulations for four of these examples the convex quadratically constrained quadratic programming qcqp problem problems involving fractional quadratic functions rutcor rutgers university e mail alizadeh rutcor rutgers edu research supported in part by the u s national science foundation grant ccr 9901991 ieor columbia university e mail gold ieor columbia edu research supported in part by the department of energy grant de fg02 92er25126 national science foundation grants dms 94 14438 cda 97 26385 and dms 01 04282

a priori error estimates for finite element methods with numerical quadrature for nonmonotone nonlinear elliptic problems the effect of numerical quadrature in finite element methods for solving quasilinear elliptic problems of nonmonotone type is studied under similar assumption on the quadrature formula as for linear problems optimal error estimates in the l 2 and the h 1 norms are proved the numerical solution obtained from the finite element method with quadrature formula is shown to be unique for a sufficiently fine mesh the analysis is valid for both simplicial and rectangular finite elements of arbitrary order numerical experiments corroborate the theoretical convergence rates

finite element approximation of a problem with a nonlinear newton boundary condition the paper is concerned with the study of an elliptic boundary value problem with a nonlinear newton boundary condition the existence and uniqueness of the solution of the continuous problem is established with the aid of the monotone operator theory the main attention is paid to the investigation of the finite element approximation using numerical integration for the computation of nonlinear boundary integrals the solvability of the discrete finite element problem is proved and the convergence of the approximate solutions to the exact one is analysed

support vector echo state machine for chaotic time series prediction a novel chaotic time series prediction method based on support vector machines svms and echo state mechanisms is proposed the basic idea is replacing kernel trick with reservoir trick in dealing with nonlinearity that is performing linear support vector regression svr in the high dimension reservoir state space and the solution benefits from the advantages from structural risk minimization principle and we call it support vector echo state machines svesms svesms belong to a special kind of recurrent neural networks rnns with convex objective function and their solution is global optimal and unique svesms are especially efficient in dealing with real life nonlinear time series and its generalization ability and robustness are obtained by regularization operator and robust loss function the method is tested on the benchmark prediction problem of mackey glass time series and applied to some real life time series such as monthly sunspots time series and runoff time series of the yellow river and the prediction results are promising

gaussian processes for object categorization discriminative methods for visual object category recognition are typically non probabilistic predicting class labels but not directly providing an estimate of uncertainty gaussian processes gps provide a framework for deriving regression techniques with explicit uncertainty models we show here how gaussian processes with covariance functions defined based on a pyramid match kernel pmk can be used for probabilistic object category recognition our probabilistic formulation provides a principled way to learn hyperparameters which we utilize to learn an optimal combination of multiple covariance functions it also offers confidence estimates at test points and naturally allows for an active learning paradigm in which points are optimally selected for interactive labeling we show that with an appropriate combination of kernels a significant boost in classification performance is possible further our experiments indicate the utility of active learning with probabilistic predictive models especially when the amount of training data labels that may be sought for a category is ultimately very small

finite element solution of flow problems with trailing conditions abstract the paper deals with the finite element solution of stream function problems describing nonviscous subsonic irrotational flows past profiles the main emphasis is laid on the treatment of the nonstandard trailing stagnation conditions which lead to physically admissible solutions the paper presents a general conception of stream function finite element modelling of complicated flow problems and a complete theory of the finite element approximations including the investigation of the existence and uniqueness of the solution of the nonsymmetric discrete problem and the convergence of approximate solutions to the exact solution

efficient svm training using low rank kernel representations svm training is a convex optimization problem which scales with the training set size rather than the feature space dimension while this is usually considered to be a desired quality in large scale problems it may cause training to be impractical the common techniques to handle this difficulty basically build a solution by solving a sequence of small scale subproblems our current effort is concentrated on the rank of the kernel matrix as a source for further enhancement of the training procedure we first show that for a low rank kernel matrix it is possible to design a better interior point method ipm in terms of storage requirements as well as computational complexity we then suggest an efficient use of a known factorization technique to approximate a given kernel matrix by a low rank matrix which in turn will be used to feed the optimizer finally we derive an upper bound on the change in the objective function value based on the approximation error and the number of active constraints support vectors this bound is general in the sense that it holds regardless of the approximation method

computation of limit and shakedown loads using a node based smoothed finite element method summary this paper presents a novel numerical procedure for computing limit and shakedown loads of structures using a node based smoothed fem in combination with a primal dual algorithm an associated primal dual form based on the von mises yield criterion is adopted the primal dual algorithm together with a newton like iteration are then used to solve this associated primal dual form to determine simultaneously both approximate upper and quasi lower bounds of the plastic collapse limit and the shakedown limit the present formulation uses only linear approximations and its implementation into finite element programs is quite simple several numerical examples are given to show the reliability accuracy and generality of the present formulation compared with other available methods copyright 2011 john wiley sons ltd

estimating the support of a high dimensional distribution suppose you are given some data set drawn from an underlying probability distribution p and you want to estimate a simple subset s of input space such that the probability that a test point drawn from p lies outside of s equals some a priori specified value between 0 and 1 we propose a method to approach this problem by trying to estimate a function f that is positive on s and negative on the complement the functional form of f is given by a kernel expansion in terms of a potentially small subset of the training data it is regularized by controlling the length of the weight vector in an associated feature space the expansion coefficients are found by solving a quadratic programming problem which we do by carrying out sequential optimization over pairs of input patterns we also provide a theoretical analysis of the statistical performance of our algorithm the algorithm is a natural extension of the support vector algorithm to the case of unlabeled data

on galerkin approximations of a quasilinear nonpotential elliptic problem of a nonmonotone type abstract we prove the convergence of galerkin approximations for a second order quasilinear elliptic problem of a divergence form with nonlinear mixed boundary conditions the existence and uniqueness of a weak and approximate solution is studied as well

on the finite element approximation of a cascade flow problem the finite element analysis of a cascade flow problem with a given velocity circulation round profiles is presented the nonlinear problem for the stream function with nonstandard boundary conditions is discretized by conforming linear triangular elements we deal with the properties of the discrete problem and study the convergence of the method both for polygonal and nonpolygonal domains including the effect of numerical integration

a kernel approach for semisupervised metric learning while distance function learning for supervised learning tasks has a long history extending it to learning tasks with weaker supervisory information has only been studied recently in particular some methods have been proposed for semisupervised metric learning based on pairwise similarity or dissimilarity information in this paper we propose a kernel approach for semisupervised metric learning and present in detail two special cases of this kernel approach the metric learning problem is thus formulated as an optimization problem for kernel learning an attractive property of the optimization problem is that it is convex and hence has no local optima while a closed form solution exists for the first special case the second case is solved using an iterative majorization procedure to estimate the optimal solution asymptotically experimental results based on both synthetic and real world data show that this new kernel approach is promising for nonlinear metric learning

efficient sparse generalized multiple kernel learning kernel methods have been successfully applied in various applications to succeed in these applications it is crucial to learn a good kernel representation whose objective is to reveal the data similarity precisely in this paper we address the problem of multiple kernel learning mkl searching for the optimal kernel combination weights through maximizing a generalized performance measure most mkl methods employ the norm simplex constraints on the kernel combination weights which therefore involve a sparse but non smooth solution for the kernel weights despite the success of their efficiency they tend to discard informative complementary or orthogonal base kernels and yield degenerated generalization performance alternatively imposing the norm constraint on the kernel weights will keep all the information in the base kernels this leads to non sparse solutions and brings the risk of being sensitive to noise and incorporating redundant information to tackle these problems we propose a generalized mkl gmkl model by introducing an elastic net type constraint on the kernel weights more specifically it is an mkl model with a constraint on a linear combination of the norm and the squared norm on the kernel weights to seek the optimal kernel combination weights therefore previous mkl problems based on the norm or the norm constraints can be regarded as special cases furthermore our gmkl enjoys the favorable sparsity property on the solution and also facilitates the grouping effect moreover the optimization of our gmkl is a convex optimization problem where a local solution is the global optimal solution we further derive a level method to efficiently solve the optimization problem a series of experiments on both synthetic and real world datasets have been conducted to show the effectiveness and efficiency of our gmkl

learning the kernel matrix with semidefinite programming kernel based learning algorithms work by embedding the data into a euclidean space and then searching for linear relations among the embedded data points the embedding is performed implicitly by specifying the inner products between each pair of points in the embedding space this information is contained in the so called kernel matrix a symmetric and positive semidefinite matrix that encodes the relative positions of all points specifying this matrix amounts to specifying the geometry of the embedding space and inducing a notion of similarity in the input space classical model selection problems in machine learning in this paper we show how the kernel matrix can be learned from data via semidefinite programming sdp techniques when applied to a kernel matrix associated with both training and test data this gives a powerful transductive algorithm using the labeled part of the data one can learn an embedding also for the unlabeled part the similarity between test points is inferred from training points and their labels importantly these learning problems are convex so we obtain a method for learning both the model class and the function without local minima furthermore this approach leads directly to a convex method for learning the 2 norm soft margin parameter in support vector machines solving an important open problem

soft margin multiple kernel learning multiple kernel learning mkl has been proposed for kernel methods by learning the optimal kernel from a set of predefined base kernels however the traditional l 1 rm mkl method often achieves worse results than the simplest method using the average of base kernels i e average kernel in some practical applications in order to improve the effectiveness of mkl this paper presents a novel soft margin perspective for mkl specifically we introduce an additional slack variable called kernel slack variable to each quadratic constraint of mkl which corresponds to one support vector machine model using a single base kernel we first show that l 1 rm mkl can be deemed as hard margin mkl and then we propose a novel soft margin framework for mkl three commonly used loss functions including the hinge loss the square hinge loss and the square loss can be readily incorporated into this framework leading to the new soft margin mkl objective functions many existing mkl methods can be shown as special cases under our soft margin framework for example the hinge loss soft margin mkl leads to a new box constraint for kernel combination coefficients using different hyper parameter values for this formulation we can inherently bridge the method using average kernel l 1 rm mkl and the hinge loss soft margin mkl the square hinge loss soft margin mkl unifies the family of elastic net constraint regularizer based approaches and the square loss soft margin mkl incorporates l 2 rm mkl naturally moreover we also develop efficient algorithms for solving both the hinge loss and square hinge loss soft margin mkl comprehensive experimental studies for various mkl algorithms on several benchmark data sets and two real world applications including video action recognition and event recognition demonstrate that our proposed algorithms can efficiently achieve an effective yet sparse solution for mkl

multiple kernel learning algorithms in recent years several methods have been proposed to combine multiple kernels instead of using a single one these different kernels may correspond to using different notions of similarity or may be using information coming from multiple sources different representations or different feature subsets in trying to organize and highlight the similarities and differences between them we give a taxonomy of and review several multiple kernel learning algorithms we perform experiments on real data sets for better illustration and comparison of existing algorithms we see that though there may not be large differences in terms of accuracy there is difference between them in complexity as given by the number of stored support vectors the sparsity of the solution as given by the number of used kernels and training time complexity we see that overall using multiple kernels instead of a single one is useful and believe that combining kernels in a nonlinear or data dependent way seems more promising than linear combination in fusing information provided by simple linear kernels whereas linear methods are more reasonable when combining complex gaussian kernels

efficient hyperkernel learning using second order cone programming the kernel function plays a central role in kernel methods most existing methods can only adapt the kernel parameters or the kernel matrix based on empirical data recently ong et al introduced the method of hyperkernels which can be used to learn the kernel function directly in an inductive setting however the associated optimization problem is a semidefinite program sdp which is very computationally expensive even with the recent advances in interior point methods in this paper we show that this learning problem can be equivalently reformulated as a second order cone program socp which can then be solved more efficiently than sdps comparison is also made with the kernel matrix learning method proposed by lanckriet et al experimental results on both classification and regression problems with toy and real world data sets show that our proposed socp formulation has significant speedup over the original sdp formulation moreover it yields better generalization than lanckriet et al s method with a speed that is comparable or sometimes even faster than their quadratically constrained quadratic program qcqp formulation