large scale multiple kernel learning while classical kernel based learning algorithms are based on a single kernel in practice it is often desirable to use multiple kernels lanckriet et al 2004 considered conic combinations of kernel matrices for classification leading to a convex quadratically constrained quadratic program we show that it can be rewritten as a semi infinite linear program that can be efficiently solved by recycling the standard svm implementations moreover we generalize the formulation and our method to a larger class of problems including regression and one class classification experimental results show that the proposed algorithm works for hundred thousands of examples or hundreds of kernels to be combined and helps for automatic model selection improving the interpretability of the learning result in a second part we discuss general speed up mechanism for svms especially when used with sparse feature maps as appear for string kernels allowing us to train a string kernel svm on a 10 million real world splice data set from computational biology we integrated multiple kernel learning in our machine learning toolbox shogun for which the source code is publicly available at http www fml tuebingen mpg de raetsch projects shogun

discrete time nonlinear h control with measurement feedback abstract this paper deals with the problem of h control via measurement feedback with internal stability for discrete time nonlinear systems using the theory of discrete time dissipative systems and differential game we first present a parameterization of a family of static state feedback h nonlinear controllers then we provide sufficient conditions for the existence of a dynamic output compensator which solves the problem of disturbance attenuation with internal stability for discrete time nonlinear systems the theory developed in this paper incorporates the well known results for discrete time linear systems and generalizes some recent advances in h control of continuous time nonlinear systems to their discrete time counterparts

improved multi kernel classification machine with nystrom approximation technique kernelized modification of ho kashyap algorithm with squared approximation of the misclassification errors kmhks is an effective algorithm for nonlinearly separable classification problems while kmhks only adopts one kernel function so a multi kernel classification machine with reduced complexity named nystrom approximation matrix with multiple kmhkss nmkmhks has been developed but nmkmhks has to initialize many parameters and has not an ability to deal with noise well to this end we propose an improved multi kernel classification machine with nystrom approximation technique inmkmhks inmkmhks is based on a new way of generating kernel functions and a new nystrom approximation technique the contributions of inmkmhks are that 1 avoiding the problem of setting too many parameters 2 keeping comparable space and computational complexities after comparing with nmkmhks 3 having a tighter generalization risk bound in terms of rademacher complexity analysis 4 having a better recognition than nmkmhks on average 5 possessing an ability to deal with noise and practical images highlightsavoiding the problem of setting some initial parameters keeping a comparable space and computational complexities having a tighter generalization risk bound having a better recognition than nmkmhks on average possessing an ability to deal with noise and practical images

state space approach to discrete time h control abstract state space solutions to a discrete time problem are given for a given number 0we give a characterization of all controllers such that the norm of the closed loop transfer function is less than the approach taken is based on the solution of two riccati equations using the stable deflating subspace of a symplectic pencil this ensures that no unnecessary assumptions on the a matrix on the realization of the transfer function are needed while the results are the direct analogues of those in the continuous time the state space derivation is considerably more difficult

an algebraic approach to h output feedback control problems abstract we derive an output feedback controller which stabilizes a system and satisfies a prescribed h norm bound of the closed loop transfer function the proposed design method is available for any system i e there are no restrictions on d12 and d21 this approach utilizes only algebraic operations thus proofs are simple and clear

human activity recognition using multi features and multiple kernel learning this paper presents two sets of features shape representation and kinematic structure for human activity recognition using a sequence of rgb d images the shape features are extracted using the depth information in the frequency domain via spherical harmonics representation the other features include the motion of the 3d joint positions i e the end points of the distal limb segments in the human body both sets of features are fused using the multiple kernel learning mkl technique at the kernel level for human activity recognition our experiments on three publicly available datasets demonstrate that the proposed features are robust for human activity recognition and particularly when there are similarities among the actions

disturbance attenuation and h sub infinity control via measurement feedback in nonlinear systems a solution to the problem of disturbance attenuation via measurement feedback with internal stability is presented for an affine nonlinear system it is shown that the concept of disturbance attenuation in the sense of truncated l sub 2 norms can be given an interpretation in terms of the response to periodic inputs in the sense of rms amplitude even in the nonlinear setup in the case of state feedback a family of controllers is also provided the proofs of all these results are simple and provide deeper insight even in the analysis of the corresponding linear control problem

generalized hierarchical kernel learning this paper generalizes the framework of hierarchical kernel learning hkl and illustrates its utility in the domain of rule learning hkl involves multiple kernel learning over a set of given base kernels assumed to be embedded on a directed acyclic graph this paper proposes a two fold generalization of hkl the first is employing a generic l1 lp block norm regularizer 1 2 that alleviates a key limitation of the hkl formulation the second is a generalization to the case of multi class multi label and more generally multi task applications the main technical contribution of this work is the derivation of a highly specialized partial dual of the proposed generalized hkl formulation and an efficient mirror descent based active set algorithm for solving it importantly the generic regularizer enables the proposed formulation to be employed in the rule ensemble learning rel where the goal is to construct an ensemble of conjunctive propositional rules experiments on benchmark rel data sets illustrate the efficacy of the proposed generalizations

localized algorithms for multiple kernel learning instead of selecting a single kernel multiple kernel learning mkl uses a weighted sum of kernels where the weight of each kernel is optimized during training such methods assign the same weight to a kernel over the whole input space and we discuss localized multiple kernel learning lmkl that is composed of a kernel based learning algorithm and a parametric gating model to assign local weights to kernel functions these two components are trained in a coupled manner using a two step alternating optimization algorithm empirical results on benchmark classification and regression data sets validate the applicability of our approach we see that lmkl achieves higher accuracy compared with canonical mkl on classification problems with different feature representations lmkl can also identify the relevant parts of images using the gating model as a saliency detector in image recognition problems in regression tasks lmkl improves the performance significantly or reduces the model complexity by storing significantly fewer support vectors

new approach to general nonlinear discrete time stochastic h_ infty control in this paper a new approach based on convex analysis is introduced to solve the h infty problem for discrete time nonlinear stochastic systems first by using the disintegration property of the conditional expectation sufficient necessary conditions are given for the internal stability of the concerned systems second in order to separate the unknown exogenous disturbance from the state variables the properties of convex functions are applied to prove the stochastic version of bounded real lemma brl third the state feedback h infty control problem is studied and an h infty control is designed based on the brl finally two numerical examples and one real world regulation control example of synthetic genetic circuit are presented to show the effectiveness of our developed theory

extreme sojourns of a gaussian process with a point of maximum variance letx t 0 t 1 be a gaussian process with mean 0 and continuous covariance function put 2 t ex 2 t and suppose that there is a point in 0 1 such that 2 t has unique maximum att putl u mes s 0 s 1 x s u under suitable conditions there exists a functionv v u and a distribution functiong x x 0 such thatv u foru and mathop lim limits u to infty frac int limits 0 x ydp upsilon l u leqq y upsilon el u g x

passivity and absolute stabilization of a class of discrete time nonlinear systems abstract by using the theory of discrete time passive systems and the concept of feedback equivalence we present an approach toward deriving sufficient conditions for the global stabilization of discrete time nonlinear systems in the form x k 1 tf x k g x k u k the central feature of the approach is that a unifying set of stability criteria can be obtained and the stability results recover the previous stabilization schemes proposed in the literature

success based locally weighted multiple kernel combination base kernels have local regions of success ros or expertise in feature space ros for each base kernel is identified during training from cross validation set these ros are modeled using regression in terms of success prediction functions spf spfs are used as instance dependent weighing functions of base kernels in mkl framework proposed weighing scheme maximizes alignment with ideal kernel multiple kernel learning mkl literature has mostly focused on learning weights for base kernel combiners recent works using instance dependent weights have resulted in better performance compared to fixed weight mkl approaches this may be attributed to the fact that different base kernels have varying discriminative capabilities in distinct local regions of input space we refer to the zones of classification expertize of base kernels as their regions of success ros we propose to identify and model them during training through a set of instance dependent success prediction functions spf having high values in ros and low otherwise during operation the use of these spfs as instance dependent weighing functions promotes locally discriminative base kernels while suppressing others we have experimented with 21 benchmark datasets from various domains having large variations in terms of dataset size interclass imbalances and number of features our proposal has achieved higher classification rates and balanced performance for both positive and negative classes compared to other instance dependent and fixed weight approaches

losslessness feedback equivalence and the global stabilization of discrete time nonlinear systems in this paper a necessary and sufficient condition for a nonlinear system of the form spl sigma given by x k 1 f x k g x k u k y k h x k j x k u k to be lossless is given and it is shown that a lossless system can be globally asymptotically stabilized by output feedback if and only if the system is zero state observable then we investigate conditions under which spl sigma can be rendered lossless via smooth state feedback in particular we show that this is possible if and only if the system in question has relative degree spl lcub 0 0 spl rcub and has lossless zero dynamics under suitable controllability like rank conditions we prove that nonlinear systems having relative degree spl lcub 0 0 spl rcub and lossless zero dynamics can be globally stabilized by smooth state feedback as a consequence we obtain sufficient conditions for a class of cascaded systems to be globally stabilizable the global stabilization problem of the nonlinear system spl sigma without output is also investigated in this paper by means of feedback equivalence some of the results are parallel to analogous ones in continuous time but in many respects the theory is substantially different and many new phenomena appear

on a state space approach to nonlinear h control we study the standard h optimal control problem using state feedback for smooth nonlinear control systems the main theorem obtained roughly states that the l2 induced norm from disturbances to inputs and outputs can be made smaller than a constant 0 if the corresponding h norm for the system linearized at the equilibrium can be made smaller than by linear state feedback necessary and sufficient conditions for the latter problem are by now well known e g from the state space approach to linear h optimal control our approach to the nonlinear h optimal control problem generalizes the state space approach to the linear h problem by replacing the hamiltonian matrix and corresponding riccati equation as used in the linear context by a hamiltonian vector field together with a hamiltonian jacobi equation corresponding to its stable invariant manifold

h sub spl infin control of discrete time nonlinear systems this paper presents an explicit solution to the problem of disturbance attenuation with internal stability via full information feedback state feedback and dynamic output feedback respectively for discrete time nonlinear systems the h sub spl infin control theory is first developed for affine systems and then extended to general nonlinear systems based on the concepts of dissipation inequality differential game and lasalle s invariance principle in discrete time a substantial difficulty that v a x b x u e x w respectively v f x u w is no longer quadratic in sub w sup u arising in the case of discrete time nonlinear systems has been surmounted in the paper in the case of a linear system we show how the results reduce to the well known ones recently proposed in the literature

a transfer function approach to the problems of discrete time systems h sub infinity optimal linear control and filtering a solution is derived to the problems of h sub infinity optimal linear state regulation and filtering the solution method for both problems is based on a transfer function approach which applies standard spectral factorization return difference relations are given which are extensions of the relations associated with the linear quadratic problem

on the discrete time bounded real lemma with application in the characterization of static state feedback h controllers abstract this paper deals with the discrete time bounded real lemma and its application in the characterization of all static state feedback h controllers for discrete time systems we first present two versions of the discrete time bounded real lemma the results apply to non minimal stable state space realizations and are given in terms of either the positive semi definite strong solution or the positive semi definite stabilizing solution of a discrete algebraic riccati equation based on the discrete time bounded real lemma we obtain a characterization of all static state feedback h controllers for discrete time systems as an application of the parameterization of all static state feedback h controllers we develop a solution to the discrete time h control problem via static output feedback also a numerical procedure is proposed to determine an output feedback gain

two riccati formulae for the discrete time h control problem a two riccati state space approach to the discrete time h control problem is presented the main theoretical tool is the kalman szego popov yakubovich system written in adequate forms in order to reduce the problems to a standard form for which the solutions are easily obtained the necessity part of our main results uses a signed counterpart of the positivity theorem

an evaluation of classifier specific filter measure performance for feature selection feature selection is an important part of classifier design there are many possible methods for searching and evaluating feature subsets but little consensus on which methods are best this paper examines a number of filter based feature subset evaluation measures with the goal of assessing their performance with respect to specific classifiers this work tests 16 common filter measures for use with k nearest neighbors and support vector machine classifiers the measures are tested on 20 real and 20 artificial data sets which are designed to probe for specific challenges the strengths and weaknesses of each measure are discussed with respect to the specific challenges and correlation with classifier accuracy the results highlight several challenging problems with a number of common filter measures the results indicate that the best filter measure is classifier specific k nearest neighbors classifiers work well with subset based relief correlation feature selection or conditional mutual information maximization whereas fisher s interclass separability criterion and conditional mutual information maximization work better for support vector machines despite the large number and variety of feature selection measures proposed in the literature no single measure is guaranteed to outperform the others even within a single classifier and the overall performance of a feature selection method cannot be characterized independently of the subsequent classifier highlightscompare common feature selection filter measures for use with specific classifiers many tested filter measures do not reliably predict classifier accuracy some measures have specific problems that cause them to select unsuitable features best feature selection filter measure is classifier specific

learning the kernel matrix with semidefinite programming kernel based learning algorithms work by embedding the data into a euclidean space and then searching for linear relations among the embedded data points the embedding is performed implicitly by specifying the inner products between each pair of points in the embedding space this information is contained in the so called kernel matrix a symmetric and positive semidefinite matrix that encodes the relative positions of all points specifying this matrix amounts to specifying the geometry of the embedding space and inducing a notion of similarity in the input space classical model selection problems in machine learning in this paper we show how the kernel matrix can be learned from data via semidefinite programming sdp techniques when applied to a kernel matrix associated with both training and test data this gives a powerful transductive algorithm using the labeled part of the data one can learn an embedding also for the unlabeled part the similarity between test points is inferred from training points and their labels importantly these learning problems are convex so we obtain a method for learning both the model class and the function without local minima furthermore this approach leads directly to a convex method for learning the 2 norm soft margin parameter in support vector machines solving an important open problem

an approach to h control of a class of nonlinear systems this paper considers robust h control of a class of nonlinear systems that can be represented by a heterogeneous model a new controller design method using both state feedback is proposed based on quadratic stability theory a number of necessary and sufficient conditions are derived for quadratic stabilizability with disturbance attenuation of the control system concerned an example is also given to demonstrate the performance of the proposed controller design methods

times series segmentation a sliding window approach the aim of this paper is to present two on line sliding window segmentation algorithms detection nonstationarity is based on parameter fluctuations and change point localization of the akaike information criterion asymptotic properties of the proposed algorithms are analyzed specifically the limiting distributions are derived and the asymptotic threshold values are tabulated for future reference finite sample simulations are performed to illustrate the usefulness of these algorithms

h sub spl infin control via measurement feedback for general nonlinear systems this paper shows how the problem of local disturbance attenuation via measurement feedback with internal stability can be solved for a nonlinear system of rather general structure the solution of the problem is shown to be related to the existence of solutions of a pair of hamilton jacobi inequalities in n independent variables which are associated with state feedback and respectively output injection design

edit distance based kernel functions for structural pattern classification a common approach in structural pattern classification is to define a dissimilarity measure on patterns and apply a distance based nearest neighbor classifier in this paper we introduce an alternative method for classification using kernel functions based on edit distance the proposed approach is applicable to both string and graph representations of patterns by means of the kernel functions introduced in this paper string and graph classification can be performed in an implicit vector space using powerful statistical algorithms the validity of the kernel method cannot be established for edit distance in general however by evaluating theoretical criteria we show that the kernel functions are nevertheless suitable for classification and experiments on various string and graph datasets clearly demonstrate that nearest neighbor classifiers can be outperformed by support vector machines using the proposed kernel functions

multiple kernel learning algorithms in recent years several methods have been proposed to combine multiple kernels instead of using a single one these different kernels may correspond to using different notions of similarity or may be using information coming from multiple sources different representations or different feature subsets in trying to organize and highlight the similarities and differences between them we give a taxonomy of and review several multiple kernel learning algorithms we perform experiments on real data sets for better illustration and comparison of existing algorithms we see that though there may not be large differences in terms of accuracy there is difference between them in complexity as given by the number of stored support vectors the sparsity of the solution as given by the number of used kernels and training time complexity we see that overall using multiple kernels instead of a single one is useful and believe that combining kernels in a nonlinear or data dependent way seems more promising than linear combination in fusing information provided by simple linear kernels whereas linear methods are more reasonable when combining complex gaussian kernels