consensus and complementarity based maximum entropy discrimination for multi view classification maximum entropy discrimination med is a general framework for discriminative estimation based on the maximum entropy and large margin principles but it just uses only one view of the data not all the views of the data although multi view maximum entropy discrimination considered the multi view information of the data it just respects the consensus principle there are two common principles named consensus and complementarity in multi view learning we aim to take the full advantage of the multi view information of the data for classification and respect the two common principles consensus and complementarity simultaneously in this paper we propose a new method called consensus and complementarity based med med 2c for multi view classification which well utilizes the two principles consensus and complementarity for multi view learning mvl we first transform data from two views into a common subspace and make the transformed data in the new subspace identical to respect the consensus principle then we augment the transformed data with their original features to take into account the complementarity principle built on the augmented features and meanwhile by relaxing the consensus principle the objective function of med 2c is naturally formed whose inner optimization recovers the traditional med framework we provide an instantiation of the med 2c method and derive the corresponding solution experimental results on synthetic data and real world data show the effectiveness of the proposed med 2c it not only performs better than three single view classification methods but also generally outperforms three multi view classification methods canonical correlation analysis cca ensemble med emed and the multi view variant svm 2k of the classical support vector machine svm algorithm in addition med 2c performs better than or as well as state of the art mvmed on all the data sets

information theoretic subspace clustering this paper addresses the problem of grouping the data points sampled from a union of multiple subspaces in the presence of outliers information theoretic objective functions are proposed to combine structured low rank representations lrrs to capture the global structure of data and information theoretic measures to handle outliers in theoretical part we point out that group sparsity induced measures ell 2 1 norm ell alpha norm and correntropy can be justified from the viewpoint of half quadratic hq optimization which facilitates both convergence study and algorithmic development in particular a general formulation is accordingly proposed to unify hq based group sparsity methods into a common framework in algorithmic part we develop information theoretic subspace clustering methods via correntropy with the help of parzen window estimation correntropy is used to handle either outliers under any distributions or sample specific errors in data pairwise link constraints are further treated as a prior structure of lrrs based on the hq framework iterative algorithms are developed to solve the nonconvex information theoretic loss functions experimental results on three benchmark databases show that our methods can further improve the robustness of lrr subspace clustering and outperform other state of the art subspace clustering methods

manifold regularization a geometric framework for learning from labeled and unlabeled examples we propose a family of learning algorithms based on a new form of regularization that allows us to exploit the geometry of the marginal distribution we focus on a semi supervised framework that incorporates labeled and unlabeled data in a general purpose learner some transductive graph learning algorithms and standard methods including support vector machines and regularized least squares can be obtained as special cases we use properties of reproducing kernel hilbert spaces to prove new representer theorems that provide theoretical basis for the algorithms as a result in contrast to purely graph based approaches we obtain a natural out of sample extension to novel examples and so are able to handle both transductive and truly semi supervised settings we present experimental evidence suggesting that our semi supervised algorithms are able to use unlabeled data effectively finally we have a brief discussion of unsupervised and fully supervised learning within our general framework

on the representation of operators in bases of compactly supported wavelets this paper describes exact and explicit representations of the differential operators d n dx n n 1 2 cdots in orthonormal bases of compactly supported wavelets as well as the representations of the hilbert transform and fractional derivatives the method of computing these representations is directly applicable to multidimensional convolution operators also sparse representations of shift operators in orthonormal bases of compactly supported wavelets are discussed and a fast algorithm requiring o n log n operations for computing the wavelet coefficients of all n circulant shifts of a vector of the length n 2 n is constructed as an example of an application of this algorithm it is shown that the storage requirements of the fast algorithm for applying the standard form of a pseudodifferential operator to a vector see g beylkin r r coifman and v rokhlin comm pure appl math 44 1991 pp 141 183 may be reduced from o n to o log 2 n significant entries

semisupervised support vector machines with tangent space intrinsic manifold regularization semisupervised learning has been an active research topic in machine learning and data mining one main reason is that labeling examples is expensive and time consuming while there are large numbers of unlabeled examples available in many practical problems so far laplacian regularization has been widely used in semisupervised learning in this paper we propose a new regularization method called tangent space intrinsic manifold regularization it is intrinsic to data manifold and favors linear functions on the manifold fundamental elements involved in the formulation of the regularization are local tangent space representations which are estimated by local principal component analysis and the connections that relate adjacent tangent spaces simultaneously we explore its application to semisupervised classification and propose two new learning algorithms called tangent space intrinsic manifold regularized support vector machines tisvms and tangent space intrinsic manifold regularized twin svms titsvms they effectively integrate the tangent space intrinsic manifold regularization consideration the optimization of tisvms can be solved by a standard quadratic programming while the optimization of titsvms can be solved by a pair of standard quadratic programmings the experimental results of semisupervised classification problems show the effectiveness of the proposed semisupervised learning algorithms

adaptive solution of partial differential equations in multiwavelet bases we construct multiresolution representations of derivative and exponential operators with linear boundary conditions in multiwavelet bases and use them to develop a simple adaptive scheme for the solution of nonlinear time dependent partial differential equations the emphasis on hierarchical representations of functions on intervals helps to address issues of both high order approximation and efficient application of integral operators and the lack of regularity of multiwavelets does not preclude their use in representing differential operators comparisons with finite difference finite element and spectral element methods are presented as are numerical examples with the heat equation and burgers equation

high order splitting methods for the incompressible navier stokes equations abstract a new pressure formulation for splitting methods is developed that results in high order time accurate schemes for the solution of the incompressible navier stokes equations in particular improved pressure boundary conditions of high order in time are introduced that minimize the effect of erroneous numerical boundary layers induced by splitting methods a new family of stiffly stable schemes is employed in mixed explicit implicit time intgration rules these schemes exhibit much broader stability regions as compared to adams family schemes typically used in splitting methods their stability properties remain almost constant as the accuracy of the integration increases so that robust third or higher order time accurate schemes can readily be constructed that remain stable at relatively large cfl number the new schemes are implemented within the framework of spectral element discretizations in space so that flexibility and accuracy is guaranteed in the numerical experimentation a model stokes problem is studied in detail and several examples of navier stokes solutions of flows in complex geometries are reported comparison is made with the previously used first order in time spectral element splitting and non splitting e g uzawa schemes high order splitting spectral element methods combine accuracy in space and time and flexibility in geometry and thus can be very efficient in direct simulations of turbulent flows in complex geometries

on the adaptive numerical solution of nonlinear partial differential equations in wavelet bases this work develops fast and adaptive algorithms for numerically solving nonlinear partial differential equations of the formut lu nf u where l and n are linear differential operators andf u is a nonlinear function these equations are adaptively solved by projecting the solutionuand the operators l and n into a wavelet basis vanishing moments of the basis functions permit a sparse representation of the solution and operators using these sparse representations fast and adaptive algorithms that apply operators to functions and evaluate nonlinear functions are developed for solving evolution equations for a wavelet representation of the solutionuthat containsnssignificant coefficients the algorithms update the solution usingo ns operations the approach is applied to a number of examples and numerical results are given

alternative multiview maximum entropy discrimination maximum entropy discrimination med is a general framework for discriminative estimation based on maximum entropy and maximum margin principles and can produce hard margin support vector machines under some assumptions recently the multiview version of med multiview med mvmed was proposed in this paper we try to explore a more natural mvmed framework by assuming two separate distributions p 1 theta 1 over the first view classifier parameter theta 1 and p 2 theta 2 over the second view classifier parameter theta 2 we name the new mvmed framework as alternative mvmed amvmed which enforces the posteriors of two view margins to be equal the proposed amvmed is more flexible than the existing mvmed because compared with mvmed which optimizes one relative entropy amvmed assigns one relative entropy term to each of the two views thus incorporating a tradeoff between the two views we give the detailed solving procedure which can be divided into two steps the first step is solving our optimization problem without considering the equal margin posteriors from two views and then in the second step we consider the equal posteriors experimental results on multiple real world data sets verify the effectiveness of the amvmed and comparisons with mvmed are also reported

noise robust semi supervised learning via fast sparse coding this paper presents a novel noise robust graph based semi supervised learning algorithm to deal with the challenging problem of semi supervised learning with noisy initial labels inspired by the successful use of sparse coding for noise reduction we choose to give new l1 norm formulation of laplacian regularization for graph based semi supervised learning since our l1 norm laplacian regularization is explicitly defined over the eigenvectors of the normalized laplacian matrix we formulate graph based semi supervised learning as an l1 norm linear reconstruction problem which can be efficiently solved by sparse coding furthermore by working with only a small subset of eigenvectors we develop a fast sparse coding algorithm for our l1 norm semi supervised learning finally we evaluate the proposed algorithm in noise robust image classification the experimental results on several benchmark datasets demonstrate the promising performance of the proposed algorithm highlightswe propose novel semi supervised learning based on fast sparse coding our algorithm achieves promising results in noise robust image classification our algorithm can readily be extended to many other challenging problems

conjugate relation between loss functions and uncertainty sets in classification problems there are two main approaches to binary classification problems the loss function approach and the uncertainty set approach the loss function approach is widely used in real world data analysis statistical decision theory has been used to elucidate its properties such as statistical consistency conditional probabilities can also be estimated by using the minimum solution of the loss function in the uncertainty set approach an uncertainty set is defined for each binary label from training samples the best separating hyperplane between the two uncertainty sets is used as the decision function although the uncertainty set approach provides an intuitive understanding of learning algorithms its statistical properties have not been sufficiently studied in this paper we show that the uncertainty set is deeply connected with the convex conjugate of a loss function on the basis of the conjugate relation we propose a way of revising the uncertainty set approach so that it will have good statistical properties such as statistical consistency we also introduce statistical models corresponding to uncertainty sets in order to estimate conditional probabilities finally we present numerical experiments verifying that the learning with revised uncertainty sets improves the prediction accuracy

value regularization and fenchel duality regularization is an approach to function learning that balances fit and smoothness in practice we search for a function f with a finite representation f i ci i in most treatments the ci are the primary objects of study we consider value regularization constructing optimization problems in which the predicted values at the training points are the primary variables and therefore the central objects of study although this is a simple change it has profound consequences from convex conjugacy and the theory of fenchel duality we derive separate optimality conditions for the regularization and loss portions of the learning problem this technique yields clean and short derivations of standard algorithms this framework is ideally suited to studying many other phenomena at the intersection of learning theory and optimization we obtain a value based variant of the representer theorem which underscores the transductive nature of regularization in reproducing kernel hilbert spaces we unify and extend previous results on learning kernel functions with very simple proofs we analyze the use of unregularized bias terms in optimization problems and low rank approximations to kernel matrices obtaining new results in these areas in summary the combination of value regularization and fenchel duality are valuable tools for studying the optimization problems in machine learning

fast wavelet transforms and numerical algorithms i a class of algorithms is introduced for the rapid numerical application of a class of linear operators to arbitrary vectors previously published schemes of this type utilize detailed analytical information about the operators being applied and are specific to extremely narrow classes of matrices in contrast the methods presented here are based on the recently developed theory of wavelets and are applicable to all calderon zygmund and pseudo differential operators the algorithms of this paper require order o n or o n log n operations to apply an n n matrix to a vector depending on the particular operator and the version of the algorithm being used and our numerical experiments indicate that many previously intractable problems become manageable with the techniques presented here

the spectral element method for the shallow water equations on the sphere the spectral element method is implemented for the shallow water equations in spherical geometry and its performance is compared with other models this is the first step in evaluating the suitability of spectral elements for climate modeling the potential advantages and disadvantages of spectral elements over more conventional models used for climate studies are discussed the method requires that the sphere be tiled with rectangles for which we make use of the gnomonic projection to map the sphere onto the cube to measure the performance of the method relative to other models results are presented from a standard suite of shallow water test cases for the sphere these results confirm the spectral accuracy of the method

modified spectral parameter power series representations for solutions of sturm liouville equations and their applications abstract spectral parameter power series spps representations for solutions of sturm liouville equations proved to be an efficient practical tool for solving corresponding spectral and scattering problems they are based on a computation of recursive integrals sometimes called formal powers in this paper new relations between the formal powers are presented which considerably improve and extend the application of the spps method for example originally the spps method at a first step required to construct a nonvanishing in general a complex valued particular solution corresponding to the zero value of the spectral parameter the obtained relations remove this limitation additionally equations with nasty sturm liouville coefficients 1 p or r can be solved by the spps method we develop the spps representations for solutions of sturm liouville equations of the form p x u q x u k 1 n k r k u x a b where r k u r k x u s k x u k 1 n the complex valued functions p q r k s k are continuous on the finite segment a b several numerical examples illustrate the efficiency of the method and its wide applicability

orthonormal bases of compactly supported wavelets we construct orthonormal bases of compactly supported wavelets with arbitrarily high regularity the order of regularity increases linearly with the support width we start by reviewing the concept of multiresolution analysis as well as several algorithms in vision decomposition and reconstruction the construction then follows from a synthesis of these different approaches

sparse semi supervised learning using conjugate functions in this paper we propose a general framework for sparse semi supervised learning which concerns using a small portion of unlabeled data and a few labeled data to represent target functions and thus has the merit of accelerating function evaluations when predicting the output of a new example this framework makes use of fenchel legendre conjugates to rewrite a convex insensitive loss involving a regularization with unlabeled data and is applicable to a family of semi supervised learning methods such as multi view co regularized least squares and single view laplacian support vector machines svms as an instantiation of this framework we propose sparse multi view svms which use a squared e insensitive loss the resultant optimization is an inf sup problem and the optimal solutions have arguably saddle point properties we present a globally optimal iterative algorithm to optimize the problem we give the margin bound on the generalization error of the sparse multi view svms and derive the empirical rademacher complexity for the induced function class experiments on artificial and real world data show their effectiveness we further give a sequential training approach to show their possibility and potential for uses in large scale problems and provide encouraging experimental results indicating the efficacy of the margin bound and empirical rademacher complexity on characterizing the roles of unlabeled data for semi supervised learning

a new class of time discretization schemes for the solution of nonlinear pdes we consider issues of stability of time discretization schemes withexacttreatment of thelinear part elp schemes for solving nonlinear pdes a distinctive feature of elp schemes is the exact evaluation of the contribution of the linear term that is if the nonlinear term of the equation is zero then the scheme reduces to the evaluation of the exponential function of the operator representing the linear term computing and applying the exponential or other functions of operators with variable coefficients in the usual manner requires evaluating dense matrices and is highly inefficient it turns out that computing the exponential of strictly elliptic operators in the wavelet system of coordinates yields sparse matrices for a finite but arbitrary accuracy this observation makes our approach practical in a number of applications in particular we consider applications of elp schemes to advection diffusion equations we study the stability of these schemes and show that both explicit and implicit elp schemes have distinctly different stability properties if compared with known implicit explicit schemes for example we describe explicit schemes with stability regions similar to those of typical implicit schemes used for solving advection diffusion equations

a multiresolution approach to regularization of singular operators and fast summation singular and hypersingular operators are ubiquitous in problems of physics and their use requires a careful numerical interpretation although analytical methods for their regularization have long been known the classical approach does not provide numerical procedures for constructing or applying the regularized operator we present a multiresolution definition of regularization for integral operators with convolutional kernels which are homogeneous or associated homogeneous functions we show that our procedure yields the same operator as the classical definition moreover due to the constructive nature of our definition we provide concise numerical procedures for the construction and application of singular and hypersingular operators as an application we present an algorithm for fast computation of discrete sums and briefly discuss several other examples