kernel based hard clustering methods in the feature space with automatic variable weighting abstract this paper presents variable wise kernel hard clustering algorithms in the feature space in which dissimilarity measures are obtained as sums of squared distances between patterns and centroids computed individually for each variable by means of kernels the methods proposed in this paper are supported by the fact that a kernel function can be written as a sum of kernel functions evaluated on each variable separately the main advantage of this approach is that it allows the use of adaptive distances which are suitable to learn the weights of the variables on each cluster providing a better performance moreover various partition and cluster interpretation tools are introduced experiments with synthetic and benchmark datasets show the usefulness of the proposed algorithms and the merit of the partition and cluster interpretation tools

probability product kernels the advantages of discriminative learning algorithms and kernel machines are combined with generative modeling using a novel kernel between distributions in the probability product kernel data points in the input space are mapped to distributions over the sample space and a general inner product is then evaluated as the integral of the product of pairs of distributions the kernel is straightforward to evaluate for all exponential family models such as multinomials and gaussians and yields interesting nonlinear kernels furthermore the kernel is computable in closed form for latent distributions such as mixture models hidden markov models and linear dynamical systems for intractable models such as switching linear dynamical systems structured mean field approximations can be brought to bear on the kernel evaluation for general distributions even if an analytic expression for the kernel is not feasible we show a straightforward sampling method to evaluate it thus the kernel permits discriminative learning methods including support vector machines to exploit the properties metrics and invariances of the generative models we infer from each datum experiments are shown using multinomial models for text hidden markov models for biological data sets and linear dynamical systems for time series data

eigenfaces vs fisherfaces recognition using class specific linear projection we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression taking a pattern classification approach we consider each pixel in an image as a coordinate in a high dimensional space we take advantage of the observation that the images of a particular face under varying illumination but fixed pose lie in a 3d linear subspace of the high dimensional image space if the face is a lambertian surface without shadowing however since faces are not truly lambertian surfaces and do indeed produce self shadowing images will deviate from this linear subspace rather than explicitly modeling this deviation we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation our projection method is based on fisher s linear discriminant and produces well separated classes in a low dimensional subspace even under severe variation in lighting and facial expressions the eigenface technique another method based on linearly projecting the image space to a low dimensional subspace has similar computational requirements yet extensive experimental results demonstrate that the proposed fisherface method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases

analyticity of periodic traveling free surface water waves with vorticity we prove that the prole of a periodic traveling wave propagating at the surface of water above a at bed in a ow with a real analytic vorticity must be real analytic provided the wave speed exceeds the horizontal uid velocity throughout the ow the real analyticity of each streamline be

improved discriminate analysis for high dimensional data and its application to face recognition many pattern recognition applications involve the treatment of high dimensional data and the small sample size problem principal component analysis pca is a common used dimension reduction technique linear discriminate analysis lda is often employed for classification pca plus lda is a famous framework for discriminant analysis in high dimensional space and singular cases in this paper we examine the theory of this framework and find out that even if there is no small sample size problem the pca dimension reduction cannot guarantee the subsequent successful application of lda we thus develop an improved discriminate analysis method by introducing an inverse fisher criterion and adding a constrain in pca procedure so that the singularity phenomenon will not occur experiment results on face recognition suggest that this new approach works well and can be applied even when the number of training samples is one per class

constructing descriptive and discriminative nonlinear features rayleigh coefficients in kernel feature spaces we incorporate prior knowledge to construct nonlinear algorithms for invariant feature extraction and discrimination employing a unified framework in terms of a nonlinearized variant of the rayleigh coefficient we propose nonlinear generalizations of fisher s discriminant and oriented pca using support vector kernel functions extensive simulations show the utility of our approach

an efficient kernel matrix evaluation measure we study the problem of evaluating the goodness of a kernel matrix for a classification task as kernel matrix evaluation is usually used in other expensive procedures like feature and model selections the goodness measure must be calculated efficiently most previous approaches are not efficient except for kernel target alignment kta that can be calculated in o n 2 time complexity although kta is widely used we show that it has some serious drawbacks we propose an efficient surrogate measure to evaluate the goodness of a kernel matrix based on the data distributions of classes in the feature space the measure not only overcomes the limitations of kta but also possesses other properties like invariance efficiency and an error bound guarantee comparative experiments show that the measure is a good indication of the goodness of a kernel matrix

flexible discriminant analysis by optimal scoring abstract fisher s linear discriminant analysis is a valuable tool for multigroup classification with a large number of predictors one can find a reduced number of discriminant coordinate functions that are optimal for separating the groups with two such functions one can produce a classification map that partitions the reduced space into regions that are identified with group membership and the decision boundaries are linear this article is about richer nonlinear classification schemes linear discriminant analysis is equivalent to multiresponse linear regression using optimal scorings to represent the groups in this paper we obtain nonparametric versions of discriminant analysis by replacing linear regression by any nonparametric regression method in this way any multiresponse regression technique such as mars or neural networks can be postprocessed to improve its classification performance

learning the kernel with hyperkernels this paper addresses the problem of choosing a kernel suitable for estimation with a support vector machine hence further automating machine learning this goal is achieved by defining a reproducing kernel hilbert space on the space of kernels itself such a formulation leads to a statistical estimation problem similar to the problem of minimizing a regularized risk functional we state the equivalent representer theorem for the choice of kernels and present a semidefinite programming formulation of the resulting optimization problem several recipes for constructing hyperkernels are provided as well as the details of common machine learning problems experimental results for classification regression and novelty detection on uci data show the feasibility of our approach

global existence and blow up phenomena for the weakly dissipative novikov equation in this paper we consider the global existence and blow up for the weakly dissipative novikov equation we firstly establish the local well posedness for the weakly dissipative novikov equation by kato s theorem then we present some blow up results finally we present the global existence of strong solutions to the weakly dissipative equation c 2011 elsevier ltd all rights reserved

global weak solutions to the novikov equation abstract the existence of global weak solutions to the cauchy problem for the novikov equation is established in the space c 0 r l 0 h 1 r without the sign condition on the initial value the limit of viscous approximations for the equation is used to establish the existence of the global weak solution the key elements in our analysis include a new one sided super bound and a new higher norm estimate on the first order derivatives of the solution

phaseless recovery using the gauss newton method in this paper we propose a gauss newton algorithm to recover an n dimensional signal from its phaseless measurements the algorithm has two stages in the first stage the algorithm obtains a good initialization by calculating the eigenvector corresponding to the largest eigenvalue of a hermitian matrix in the second stage the algorithm solves an optimization problem iteratively using the gauss newton method our initialization method makes full use of all measurements and provides a good initial guess as long as the number of random measurements is o n for real valued signals we prove that a resampled version of gauss newton iterations converges to the global optimal solution quadratically with o n log n random measurements numerical experiments show that the gauss newton method has better empirical performance than other algorithms such as the wirtinger flow algorithm and altmin phase algorithm

blow up blow up rate and decay of the solution of the weakly dissipative camassa holm equation in this paper we mainly study several problems on the weakly dissipative periodic camassa holm equation at first the local well posedness of the equation is obtained by kato s theorem a necessary and sufficient condition of the blow up of the solution and some criteria guaranteeing the blow up of the solution are established then the blow up rate of the solution is discussed moreover we prove that the equation has global solutions and these global solutions decay to zero as time goes to infinite provided the potentials associated to their initial date are of one sign

on the weakly dissipative camassa holm degasperis procesi and novikov equations abstract we show that the weakly dissipative camassa holm degasperis procesi hunter saxton and novikov equations can be reduced to their non dissipative versions by means of an exponentially time dependent scaling hence up to a simple change of variables the non dissipative and dissipative versions of these equations are equivalent similar results hold also for the equations in the so called b family of equations as well as for the two component and versions of the above equations

multiclass classifiers based on dimension reduction with generalized lda linear discriminant analysis lda has been widely used for dimension reduction of data sets with multiple classes the lda has been recently extended to various generalized lda methods that are applicable regardless of the relative sizes between the data dimension and the number of data items in this paper we propose several multiclass classifiers based on generalized lda glda algorithms taking advantage of the dimension reducing transformation matrix without requiring additional training or parameter optimization a marginal linear discriminant classifier mldc a bayesian linear discriminant classifier bldc and a one dimensional bldc are introduced for multiclass classification our experimental results illustrate that these classifiers produce higher ten fold cross validation accuracy than knn and centroid based classifiers in the reduced dimensional space obtained from glda

normalized cuts and image segmentation we propose a novel approach for solving the perceptual grouping problem in vision rather than focusing on local features and their consistencies in the image data our approach aims at extracting the global impression of an image we treat image segmentation as a graph partitioning problem and propose a novel global criterion the normalized cut for segmenting the graph the normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups we show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion we applied this approach to segmenting static images as well as motion sequences and found the results to be very encouraging

multiple kernel clustering based on centered kernel alignment multiple kernel clustering mkc which performs kernel based data fusion for data clustering is an emerging topic it aims at solving clustering problems with multiple cues most mkc methods usually extend existing clustering methods with a multiple kernel learning mkl setting in this paper we propose a novel mkc method that is different from those popular approaches centered kernel alignment an effective kernel evaluation measure is employed in order to unify the two tasks of clustering and mkl into a single optimization framework to solve the formulated optimization problem an efficient two step iterative algorithm is developed experiments on several uci datasets and face image datasets validate the effectiveness and efficiency of our mkc algorithm

on the blow up rate and the blow up set of breaking waves for a shallow water equation we consider the problem of the development of singularities for classical solutions to a new periodic shallow water equation blow up can occur only in the form of wave breaking i e the solution remains bounded but its slope becomes unbounded in finite time a quite detailed description of the wave breaking phenomenon is given there is at least a point in general depending on time where the slope becomes infinite exactly at breaking time the precise blow up rate is established and for a large class of initial data we also determine the blow up set

phase retrieval via wirtinger flow theory and algorithms we study the problem of recovering the phase from magnitude measurements specifically we wish to reconstruct a complex valued signal boldsymbol x in mathbb c n about which we have phaseless samples of the form y r left langle boldsymbol a r boldsymbol x rangle right 2 r 1 ldots m knowledge of the phase of these samples would yield a linear system this paper develops a nonconvex formulation of the phase retrieval problem as well as a concrete solution algorithm in a nutshell this algorithm starts with a careful initialization obtained by means of a spectral method and then refines this initial estimate by iteratively applying novel update rules which have low computational complexity much like in a gradient descent scheme the main contribution is that this algorithm is shown to rigorously allow the exact retrieval of phase information from a nearly minimal number of random measurements indeed the sequence of successive iterates provably converges to the solution at a geometric rate so that the proposed scheme is efficient both in terms of computational and data resources in theory a variation on this scheme leads to a near linear time algorithm for a physically realizable model based on coded diffraction patterns we illustrate the effectiveness of our methods with various experiments on image data underlying our analysis are insights for the analysis of nonconvex optimization schemes that may have implications for computational problems beyond phase retrieval

integrable peakon equations with cubic nonlinearity we present a new integrable partial differential equation found by vladimir novikov like the camassa holm and degasperis procesi equations this new equation admits peaked soliton peakon solutions but it has nonlinear terms that are cubic rather than quadratic we give a matrix lax pair for v novikov s equation and show how it is related by a reciprocal transformation to a negative flow in the sawada kotera hierarchy infinitely many conserved quantities are found as well as a bi hamiltonian structure the latter is used to obtain the hamiltonian form of the finite dimensional system for the interaction of n peakons and the two body dynamics n 2 is explicitly integrated finally all of this is compared with some analogous results for another cubic peakon equation derived by zhijun qiao

on the optimal control problem for the novikov equation with strong viscosity abstract in this paper we consider an optimal control problem for the novikov equation with strong viscosity using the faedo galerkin method we derive the existence of a unique weak solution to this equation applying lions theory we obtain the existence of an optimal solution to the control problem for this equation we also deduce the first order necessary optimality condition moreover we establish two second order sufficient optimality conditions which require coercivity of the augmented lagrangian functional on the whole space or on a suitable subspace

fisher regularized support vector machine this paper proposes fisher regularized support vector machine fishersvm fishersvm is a graph based supervised learning method fishersvm has two regularization terms maximum margin and fisher regularizations fishersvm aims to maximize the margin and minimize the within class scatter support vector machine svm and fisher discriminant analysis fda are two commonly used methods in machine learning and pattern recognition a combined method of the linear svm and fda called svm lda linear discriminant analysis has been proposed only for the linear case this paper generalizes this combined method to the nonlinear case from the view of regularization a fisher regularization is defined and incorporated into svm to obtain a fisher regularized support vector machine fishersvm in fishersvm there are two regularization terms the maximum margin regularization and fisher regularization which allow fishersvm to maximize the classification margin and minimize the within class scatter roughly speaking fishersvm can approximatively fulfill the fisher criterion and obtain good statistical separability this paper also discusses the connections of fishersvm to graph based regularization methods and the mathematical programming method to kernel fisher discriminant analysis kfda it shows that fishersvm can be thought of as a graph based supervised learning method or a robust kfda experimental results on artificial and real world data show that fishersvm has a promising generalization performance

optimizing the kernel in the empirical feature space in this paper we present a method of kernel optimization by maximizing a measure of class separability in the empirical feature space an euclidean space in which the training data are embedded in such a way that the geometrical structure of the data in the feature space is preserved employing a data dependent kernel we derive an effective kernel optimization algorithm that maximizes the class separability of the data in the empirical feature space it is shown that there exists a close relationship between the class separability measure introduced here and the alignment measure defined recently by cristianini extensive simulations are carried out which show that the optimized kernel is more adaptive to the input data and leads to a substantial sometimes significant improvement in the performance of various data classification algorithms

efficient learning of supervised kernels with a graph based loss function this paper presents our study of the problems associated with learning supervised kernels from a large amount of side information we propose a new loss function derived from the laplacian matrix of a special complete graph that is generated from the side information we analyze the relationship between the proposed loss function and the kernel alignment our theoretical analysis shows that the proposed loss function has a close relationship with kernel alignment that is they both make use of side information that is fused in a matrix in addition to a similar regularization strategy moreover the proposed loss function has a linear form and thus it is more efficient in learning side information than kernel alignment that has to be performed nonlinearly the proposed loss function is used to generate new kernels as low cost alternatives of kernels learned by certain state of the art methods the empirical results demonstrate the superiority of the proposed method over state of the art methods in terms of classification accuracy and computational cost

an efficient gaussian kernel optimization based on centered kernel polarization criterion the formulated criterion has a defined geometrical significance the approximate criterion has a determined global minimum point with weaker constraints the influence of iteration parameters on the global optimal point is slight the proposed criterion is extended to solve multiclass problems the success of kernel based learning methods is heavily dependent on the choice of a kernel function and proper setting of its parameters in this paper we optimize the gaussian kernel for binary class problems by using centered kernel polarization criterion this criterion is an extension of kernel polarization and a simplified style of centered kernel alignment compared with formulated kernel polarization criterion the proposed criterion has a defined geometrical significance and it can locate the global optimal point with less influence of threshold selection furthermore the approximate criterion function can be proved to have a determined global minimum point by adopting the euler maclaurin formula under weaker conditions in addition taking the preservation of within class local structure into account we present an evaluation criterion named local multiclass centered kernel polarization in multiclass classification scenario comparative experiments are conducted on some benchmark examples with three gaussian kernel based learning methods and the results well demonstrate the effectiveness and efficiency of the proposed quality measures

gaussian kernel optimization for pattern classification this paper presents a novel algorithm to optimize the gaussian kernel for pattern classification tasks where it is desirable to have well separated samples in the kernel feature space we propose to optimize the gaussian kernel parameters by maximizing a classical class separability criterion and the problem is solved through a quasi newton algorithm by making use of a recently proposed decomposition of the objective criterion the proposed method is evaluated on five data sets with two kernel based learning algorithms the experimental results indicate that it achieves the best overall classification performance compared with three competing solutions in particular the proposed method provides a valuable kernel optimization solution in the severe small sample size scenario

centered kernel alignment inspired fuzzy support vector machine abstract support vector machine svm is a theoretically well motivated algorithm developed from statistical learning theory which has shown impressive performance in many fields in spite of its success it still suffers from the noise sensitivity problem originating from the assumption that each training point has equal importance or weight in the training process to relax this problem the svm was extended to the fuzzy svm fsvm by applying a fuzzy membership to each training point such that different training points can make different contributions to the learning of the decision surface although well determined fuzzy memberships can improve classification performance there are no general guidelines for their construction in this paper inspired by the centered kernel alignment cka which measures the degree of similarity between two kernels or kernel matrices we propose a new fuzzy membership function calculation method in which a heuristic function derived from the cka is used to calculate the dependence between a data point and its associated label although the cka induced fsvm is similar to the kernel target alignment kta induced fsvm there is actually a critical difference without that centering the definition of alignment does not correlate well with the performance of learning machines extensive experiments are performed on real world data sets from the uci benchmark repository and the application domain of computational biology which validate the superiority of the proposed fsvm model in terms of several classification performance measures

qualitative analysis for the new shallow water model with cubic nonlinearity abstract considered in this paper is the new shallow water model with cubic nonlinearity which admits the single peaked solitons and multi peakon solutions and includes both the modified camassa holm equation also called fokas olver rosenau qiao equation and the novikov equation as two special cases the main investigation is the cauchy problem of the new shallow water model with qualitative properties of its solutions we first establish the local well posedness for the cauchy problem of the new shallow water model and then derive a precise blow up scenario and the blow up mechanisms for solutions with certain initial profiles moreover with analytic initial data we show that its solutions are analytic in both variables globally in space and locally in time finally we investigate the persistence properties of the solution to the cauchy problem of the new shallow water model

learning the kernel matrix by maximizing a kfd based class separability criterion the advantage of a kernel method often depends critically on a proper choice of the kernel function a promising approach is to learn the kernel from data automatically in this paper we propose a novel method for learning the kernel matrix based on maximizing a class separability criterion that is similar to those used by linear discriminant analysis lda and kernel fisher discriminant kfd it is interesting to note that optimizing this criterion function does not require inverting the possibly singular within class scatter matrix which is a computational problem encountered by many lda and kfd methods we have conducted experiments on both synthetic data and real world data from uci and feret showing that our method consistently outperforms some previous kernel learning methods

a support vector machine formulation to pca analysis and its kernel version in this paper we present a simple and straightforward primal dual support vector machine formulation to the problem of principal component analysis pca in dual variables by considering a mapping to a high dimensional feature space and application of the kernel trick mercer theorem kernel pca is obtained as introduced by scholkopf et al 2002 while least squares support vector machine classifiers have a natural link with the kernel fisher discriminant analysis minimizing the within class scatter around targets 1 and 1 for pca analysis one can take the interpretation of a one class modeling problem with zero target value around which one maximizes the variance the score variables are interpreted as error variables within the problem formulation in this way primal dual constrained optimization problem interpretations to the linear and kernel pca analysis are obtained in a similar style as for least square support vector machine classifiers

a new discriminant subspace analysis approach for multi class problems fukunaga koontz transform fkt is a famous feature extraction method in statistical pattern recognition which aims to find a set of vectors that have the best representative power for one class while the poorest representative power for the other class li and savvides 1 propose a one against all strategy to deal with multi class problems in which the two class fkt method can be directly applied to find the presentative vectors of each class motivated by the fkt method in this paper we propose a new discriminant subspace analysis dsa method for the multi class feature extraction problems to solve dsa we propose an iterative algorithm for the joint diagonalization jd problem finally we generalize the linear dsa method to handle nonlinear feature extraction problems via the kernel trick to demonstrate the effectiveness of the proposed method for pattern recognition problems we conduct extensive experiments on real data sets and show that the proposed method outperforms most commonly used feature extraction methods

multiple kernel fuzzy clustering while fuzzy c means is a popular soft clustering method its effectiveness is largely limited to spherical clusters by applying kernel tricks the kernel fuzzy c means algorithm attempts to address this problem by mapping data with nonlinear relationships to appropriate feature spaces kernel combination or selection is crucial for effective kernel clustering unfortunately for most applications it is uneasy to find the right combination we propose a multiple kernel fuzzy c means mkfc algorithm that extends the fuzzy c means algorithm with a multiple kernel learning setting by incorporating multiple kernels and automatically adjusting the kernel weights mkfc is more immune to ineffective kernels and irrelevant features this makes the choice of kernels less crucial in addition we show multiple kernel k means to be a special case of mkfc experiments on both synthetic and real world data demonstrate the effectiveness of the proposed mkfc algorithm

a survey of kernel and spectral methods for clustering clustering algorithms are a useful tool to explore data structures and have been employed in many disciplines the focus of this paper is the partitioning clustering problem with a special interest in two recent approaches kernel and spectral methods the aim of this paper is to present a survey of kernel and spectral clustering methods two approaches able to produce nonlinear separating hypersurfaces between clusters the presented kernel clustering methods are the kernel version of many classical clustering algorithms e g k means som and neural gas spectral clustering arise from concepts in spectral graph theory and the clustering problem is configured as a graph cut problem where an appropriate objective function has to be optimized an explicit proof of the fact that these two paradigms have the same objective is reported since it has been proven that these two seemingly different approaches have the same mathematical foundation besides fuzzy kernel clustering methods are presented as extensions of kernel k means clustering algorithm

blow up phenomenon for the integrable novikov equation abstract in this paper we investigate a new integrable equation derived recently by v s novikov generalizations of the camassa holm equation j phys a 42 34 2009 342002 14 pp analogous to the camassa holm equation and the degasperis procesi equation this new equation possesses the blow up phenomenon under the special structure of this equation we establish sufficient conditions on the initial data to guarantee the formulation of singularities in finite time a global existence result is also found

nonlinear component analysis as a kernel eigenvalue problem a new method for performing a nonlinear form of principal component analysis is proposed by the use of integral operator kernel functions one can efficiently compute principal components in high dimensional feature spaces related to input space by some nonlinear map for instance the space of all possible five pixel products in 16 16 images we give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition

in defense of one vs all classification we consider the problem of multiclass classification our main thesis is that a simple one vs all scheme is as accurate as any other approach assuming that the underlying binary classifiers are well tuned regularized classifiers such as support vector machines this thesis is interesting in that it disagrees with a large body of recent published work on multiclass classification we support our position by means of a critical review of the existing literature a substantial collection of carefully controlled experimental work and theoretical arguments

global existence and blow up phenomena for the weakly dissipative camassa holm equation abstract this paper is concerned with global existence and blow up phenomena for the weakly dissipative camassa holm equation a new global existence result and a new blow up result for strong solutions to the equation with certain profiles are presented the obtained results improve considerable the previous results

higher regularity of global attractors of a weakly dissipative fractional korteweg de vries equation in this paper we study the long time behavior of solutions to a weakly dissipative fractional korteweg de vries kdv equation on the real line r the main difficulty lies in that the dissipative term is a nonlocal operator we overcome this difficulty by the commutator estimates and product estimates associated with fractional laplacian the asymptotical compactness of solution semigroup is proved by the tail estimates finally we conclude the existence of h2 r h5 r global attractor of the weakly dissipative fractional kdv equation

minkowski metric feature weighting and anomalous cluster initializing in k means clustering this paper represents another step in overcoming a drawback of k means its lack of defense against noisy features using feature weights in the criterion the weighted k means method by huang et al 2008 2004 2005 5 7 is extended to the corresponding minkowski metric for measuring distances under minkowski metric the feature weights become intuitively appealing feature rescaling factors in a conventional k means criterion to see how this can be used in addressing another issue of k means the initial setting a method to initialize k means with anomalous clusters is adapted the minkowski metric based method is experimentally validated on datasets from the uci machine learning repository and generated sets of gaussian clusters both as they are and with additional uniform random noise features and appears to be competitive in comparison with other k means based feature weighting algorithms

on the cauchy problem for the modified novikov equation with peakon solutions abstract a new nonlinear dispersive partial differential equation with cubic nonlinearity which includes the famous novikov equation as special case is investigated we first establish the local well posedness in a range of the besov spaces b p r s p r 1 s max 3 2 1 1 p but s 2 1 p which generalize the sobolev spaces h s well posedness in h s with s 3 2 is also established by applying kato s semigroup theory then we give the precise blow up scenario moreover with analytic initial data we show that its solutions are analytic in both variables globally in space and locally in time finally we prove that peakon solutions to the equation are global weak solutions

the cauchy problem for a generalized camassa holm equation abstract in this paper we are concerned with the cauchy problem of the generalized camassa holm equation using a galerkin type approximation scheme it is shown that this equation is well posed in sobolev spaces h s s 3 2 for both the periodic and the nonperiodic case in the sense of hadamard that is the data to solution map is continuous furthermore it is proved that this dependence is sharp by showing that the solution map is not uniformly continuous the nonuniform dependence is proved using the method of approximate solutions and well posedness estimates moreover it is shown that the solution map for the generalized camassa holm equation is holder continuous in h r topology finally with analytic initial data we show that its solutions are analytic in both variables globally in space and locally in time

a robust minimax approach to classification when constructing a classifier the probability of correct classification of future data points should be maximized we consider a binary classification problem where the mean and covariance matrix of each class are assumed to be known no further assumptions are made with respect to the class conditional distributions misclassification probabilities are then controlled in a worst case setting that is under all possible choices of class conditional densities with given mean and covariance matrix we minimize the worst case maximum probability of misclassification of future data points for a linear decision boundary this desideratum is translated in a very direct way into a convex second order cone optimization problem with complexity similar to a support vector machine problem the minimax problem can be interpreted geometrically as minimizing the maximum of the mahalanobis distances to the two classes we address the issue of robustness with respect to estimation errors in the means and covariances of the classes via a simple modification of the input data we also show how to exploit mercer kernels in this setting to obtain nonlinear decision boundaries yielding a classifier which proves to be competitive with current methods including support vector machines an important feature of this method is that a worst case bound on the probability of misclassification of future data is always obtained explicitly

multiresolution gray scale and rotation invariant texture classification with local binary patterns presents a theoretically very simple yet efficient multiresolution approach to gray scale and rotation invariant texture classification based on local binary patterns and nonparametric discrimination of sample and prototype distributions the method is based on recognizing that certain local binary patterns termed uniform are fundamental properties of local image texture and their occurrence histogram is proven to be a very powerful texture feature we derive a generalized gray scale and rotation invariant operator presentation that allows for detecting the uniform patterns for any quantization of the angular space and for any spatial resolution and presents a method for combining multiple operators for multiresolution analysis the proposed approach is very robust in terms of gray scale variations since the operator is by definition invariant against any monotonic transformation of the gray scale another advantage is computational simplicity as the operator can be realized with a few operations in a small neighborhood and a lookup table experimental results demonstrate that good discrimination can be achieved with the occurrence statistics of simple rotation invariant local binary patterns

face recognition using kernel direct discriminant analysis algorithms techniques that can introduce low dimensional feature representation with enhanced discriminatory power is of paramount importance in face recognition fr systems it is well known that the distribution of face images under a perceivable variation in viewpoint illumination or facial expression is highly nonlinear and complex it is therefore not surprising that linear techniques such as those based on principle component analysis pca or linear discriminant analysis lda cannot provide reliable and robust solutions to those fr problems with complex face variations in this paper we propose a kernel machine based discriminant analysis method which deals with the nonlinearity of the face patterns distribution the proposed method also effectively solves the so called small sample size sss problem which exists in most fr tasks the new algorithm has been tested in terms of classification error rate performance on the multiview umist face database results indicate that the proposed methodology is able to achieve excellent performance with only a very small set of features being used and its error rate is approximately 34 and 48 of those of two other commonly used kernel fr approaches the kernel pca kpca and the generalized discriminant analysis gda respectively

optimized data fusion for kernel k means clustering this paper presents a novel optimized kernel k means algorithm okkc to combine multiple data sources for clustering analysis the algorithm uses an alternating minimization framework to optimize the cluster membership and kernel coefficients as a nonconvex problem in the proposed algorithm the problem to optimize the cluster membership and the problem to optimize the kernel coefficients are all based on the same rayleigh quotient objective therefore the proposed algorithm converges locally okkc has a simpler procedure and lower complexity than other algorithms proposed in the literature simulated and real life data fusion applications are experimentally studied and the results validate that the proposed algorithm has comparable performance moreover it is more efficient on large scale data sets the matlab implementation of okkc algorithm is downloadable from http homes esat kuleuven be sistawww bio syu okkc html

tuning kernel parameters for svm based on expected square distance ratio the performance of a support vector machine svm depends highly on the selection of the kernel function type and relevant parameters to choose the kernel parameters properly methods analyzing the class separability have been widely adopted for their efficiency compared with other methods such as the popular grid search algorithm this paper proposes a novel index called the expected square distance ratio esdr which can serve as a better class separability criterion than the existing ones experiments on real world datasets show that compared with common kernel parameter selection methods that utilize the between class separation the variations in esdr with respect to the kernel parameter are much more in line with those of the classification accuracy leading to better kernel parameters moreover esdr takes the exact data distribution into account and can thus be used to study the model selection problem of an svm for certain forms of data distribution as an example we employ the esdr to analyze the selection of rbf radial basis function kernel parameters for gaussian data classification

algorithms for learning kernels based on centered alignment this paper presents new and effective algorithms for learning kernels in particular as shown by our empirical results these algorithms consistently outperform the so called uniform combination solution that has proven to be difficult to improve upon in the past as well as other algorithms for learning kernels based on convex combinations of base kernels in both classification and regression our algorithms are based on the notion of centered alignment which is used as a similarity measure between kernels or kernel matrices we present a number of novel algorithmic theoretical and empirical results for learning kernels based on our notion of centered alignment in particular we describe efficient algorithms for learning a maximum alignment kernel by showing that the problem can be reduced to a simple qp and discuss a one stage algorithm for learning both a kernel and a hypothesis based on that kernel using an alignment based regularization our theoretical results include a novel concentration bound for centered alignment between kernel matrices the proof of the existence of effective predictors for kernels with high alignment both for classification and for regression and the proof of stability based generalization bounds for a broad family of algorithms for learning kernels based on centered alignment we also report the results of experiments with our centered alignment based algorithms in both classification and regression

the cauchy problem for the integrable novikov equation in this paper we consider the cauchy problem for the integrable novikov equation by using the littlewood paley decomposition and nonhomogeneous besov spaces we prove that the cauchy problem for the integrable novikov equation is locally well posed in the besov space b p r s with 1 max 1 1 p 3 2 in particular when u 0 is an element of b p r s boolean and h l with 1 max 1 1 p 3 2 for all t is an element of 0 t we have that vertical bar vertical bar u t vertical bar vertical bar h l vertical bar vertical bar u 0 vertical bar vertical bar h l we also prove that the local well posedness of the cauchy problem for the novikov equation fails in b 2 3 2 infinity c 2012 elsevier inc all rights reserved

well posedness and persistence properties for the novikov equation abstract recently novikov found a new integrable equation we call it the novikov equation in this paper which has nonlinear terms that are cubic rather than quadratic and admits peaked soliton solutions peakons firstly we prove that the cauchy problem for the novikov equation is locally well posed in the besov spaces b 2 r s which generalize the sobolev spaces h s with the critical index s 3 2 then well posedness in h s with s 3 2 is also established by applying kato s semigroup theory finally we present two results on the persistence properties of the strong solution for the novikov equation

phase retrieval using alternating minimization phase retrieval problems involve solving linear equations but with missing sign or phase for complex numbers information more than four decades after it was first proposed the seminal error reduction algorithm of gerchberg and saxton and fienup is still the popular choice for solving many variants of this problem the algorithm is based on alternating minimization i e it alternates between estimating the missing phase information and the candidate solution despite its wide usage in practice no global convergence guarantees for this algorithm are known in this paper we show that a resampling variant of this approach converges geometrically to the solution of one such problem finding a vector bf x from bf y bf a where bf y vert bf a t bf x vert and vert bf z vert denotes a vector of element wise magnitudes of bf z under the assumption that bf a is gaussian empirically we demonstrate that alternating minimization performs similar to recently proposed convex techniques for this problem which are based on lifting to a convex matrix problem in sample complexity and robustness to noise however it is much more efficient and can scale to large problems analytically for a resampling version of alternating minimization we show geometric convergence to the solution and sample complexity that is off by log factors from obvious lower bounds we also establish close to optimal scaling for the case when the unknown vector is sparse our work represents the first theoretical guarantee for alternating minimization albeit with resampling for any variant of phase retrieval problems in the non convex setting

generalizations of the camassa holm equation we classify generalized camassa holm type equations which possess infinite hierarchies of higher symmetries we show that the obtained equations can be treated as negative flows of integrable quasi linear scalar evolution equations of orders 2 3 and 5 we present the corresponding lax representations or linearization transformations for these equations some of the obtained equations seem to be new

the trajectories of particles in stokes waves analyzing a free boundary problem for harmonic functions we show that there are no closed particle paths in an irrotational inviscid traveling wave propagating at the surface of water over a flat bed within a period each particle experiences a backward forward motion with a slight forward drift

feature selection and kernel learning for local learning based clustering the performance of the most clustering algorithms highly relies on the representation of data in the input space or the hilbert space of kernel methods this paper is to obtain an appropriate data representation through feature selection or kernel learning within the framework of the local learning based clustering llc wu and scholkopf 2006 method which can outperform the global learning based ones when dealing with the high dimensional data lying on manifold specifically we associate a weight to each feature or kernel and incorporate it into the built in regularization of the llc algorithm to take into account the relevance of each feature or kernel for the clustering accordingly the weights are estimated iteratively in the clustering process we show that the resulting weighted regularization with an additional constraint on the weights is equivalent to a known sparse promoting penalty hence the weights of those irrelevant features or kernels can be shrunk toward zero extensive experiments show the efficacy of the proposed methods on the benchmark data sets

generalized discriminant analysis using a kernel approach we present a new method that we call generalized discriminant analysis gda to deal with nonlinear discriminant analysis using kernel function operator the underlying theory is close to the support vector machines svm insofar as the gda method provides a mapping of the input vectors into high dimensional feature space in the transformed space linear properties make it easy to extend and generalize the classical linear discriminant analysis lda to nonlinear discriminant analysis the formulation is expressed as an eigenvalue problem resolution using a different kernel one can cover a wide class of nonlinearities for both simulated data and alternate kernels we give classification results as well as the shape of the decision function the results are confirmed using real data to perform seed classification

efficient svm training using low rank kernel representations svm training is a convex optimization problem which scales with the training set size rather than the feature space dimension while this is usually considered to be a desired quality in large scale problems it may cause training to be impractical the common techniques to handle this difficulty basically build a solution by solving a sequence of small scale subproblems our current effort is concentrated on the rank of the kernel matrix as a source for further enhancement of the training procedure we first show that for a low rank kernel matrix it is possible to design a better interior point method ipm in terms of storage requirements as well as computational complexity we then suggest an efficient use of a known factorization technique to approximate a given kernel matrix by a low rank matrix which in turn will be used to feed the optimizer finally we derive an upper bound on the change in the objective function value based on the approximation error and the number of active constraints support vectors this bound is general in the sense that it holds regardless of the approximation method

a direct algorithm of one dimensional optimal system for the group invariant solutions a direct and systematic algorithm is proposed to find one dimensional optimal system for the group invariant solutions which is attributed to the classification of its corresponding one dimensional lie algebra since the method is based on different values of all the invariants the process itself can both guarantee the comprehensiveness and demonstrate the inequivalence of the optimal system with no further proof to leave the algorithm clear we illustrate each stage with a couple of well known examples the korteweg de vries equation and the heat equation finally we apply our method to the novikov equation and use the found optimal system to investigate the corresponding invariant solutions

an introduction to kernel based learning algorithms this paper provides an introduction to support vector machines kernel fisher discriminant analysis and kernel principal component analysis as examples for successful kernel based learning methods we first give a short background about vapnik chervonenkis theory and kernel feature spaces and then proceed to kernel based learning in supervised and unsupervised scenarios including practical and algorithmic considerations we illustrate the usefulness of kernel algorithms by discussing applications such as optical character recognition and dna analysis

kernel quadratic discriminant analysis for small sample size problem it is generally believed that quadratic discriminant analysis qda can better fit the data in practical pattern recognition applications compared to linear discriminant analysis lda method this is due to the fact that qda relaxes the assumption made by lda based methods that the covariance matrix for each class is identical however it still assumes that the class conditional distribution is gaussian which is usually not the case in many real world applications in this paper a novel kernel based qda method is proposed to further relax the gaussian assumption by using the kernel machine technique the proposed method solves the complex pattern recognition problem by combining the qda solution and the kernel machine technique and at the same time tackles the so called small sample size problem through a regularized estimation of the covariance matrix extensive experimental results indicate that the proposed method is a more sophisticated solution outperforming many traditional kernel based learning algorithms

kernel self optimization learning for kernel based feature extraction and recognition kernel learning is becoming an important research topic in the area of machine learning and it has wide applications in pattern recognition computer vision image and signal processing kernel learning provides a promising solution to nonlinear problems including nonlinear feature extraction classification and clustering however in kernel based systems the problem of the kernel function and its parameters remains to be solved methods of choosing parameters from a discrete set of values have been presented in previous studies but these methods do not change the data distribution structure in the kernel based mapping space accordingly performance is not improved because the current kernel optimization does not change the data distribution based on this problem this paper presents a uniform framework for kernel self optimization with the ability to adjust the data structure the data dependent kernel is extended and applied to kernel learning and optimization equations with two criteria for measuring data discrimination are used to solve the optimal parameter values some experiments are performed to evaluate the performance in popular kernel learning methods including kernel principal components analysis kpca kernel discriminant analysis kda and kernel locality preserving projection klpp these evaluations show that the framework of kernel self optimization is feasible for enhancing kernel based learning methods

choosing the kernel parameters for support vector machines by the inter cluster distance in the feature space determining the kernel and error penalty parameters for support vector machines svms is very problem dependent in practice a popular method to deciding the kernel parameters is the grid search method in the training process classifiers are trained with different kernel parameters and only one of the classifiers is required for the testing process this makes the training process time consuming in this paper we propose using the inter cluster distances in the feature spaces to choose the kernel parameters calculating such distance costs much less computation time than training the corresponding svm classifiers thus the proper kernel parameters can be chosen much faster experiment results show that the inter cluster distance can choose proper kernel parameters with which the testing accuracy of trained svms is competitive to the standard ones and the training time can be significantly shortened

learning the kernel matrix with semidefinite programming kernel based learning algorithms work by embedding the data into a euclidean space and then searching for linear relations among the embedded data points the embedding is performed implicitly by specifying the inner products between each pair of points in the embedding space this information is contained in the so called kernel matrix a symmetric and positive semidefinite matrix that encodes the relative positions of all points specifying this matrix amounts to specifying the geometry of the embedding space and inducing a notion of similarity in the input space classical model selection problems in machine learning in this paper we show how the kernel matrix can be learned from data via semidefinite programming sdp techniques when applied to a kernel matrix associated with both training and test data this gives a powerful transductive algorithm using the labeled part of the data one can learn an embedding also for the unlabeled part the similarity between test points is inferred from training points and their labels importantly these learning problems are convex so we obtain a method for learning both the model class and the function without local minima furthermore this approach leads directly to a convex method for learning the 2 norm soft margin parameter in support vector machines solving an important open problem

a deterministic theory for exact non convex phase retrieval in this paper we analyze the non convex framework of wirtinger flow wf for phase retrieval and identify a novel sufficient condition for universal exact recovery through the lens of low rank matrix recovery theory via a perspective in the lifted domain we show that the wf iterates converge to a true solution with fully deterministic arguments under a single condition on the lifted forward model to this end a geometric relationship between between the accuracy of spectral initialization and the validity of the regularity condition is derived in particular we determine that a certain concentration property on the spectral matrix must hold uniformly with a sufficiently tight constant this culminates into a sufficient condition that is equivalent to a restricted isometry type property over rank 1 positive semi definite matrices and amounts to a less stringent requirement on the lifted forward model than those of prominent low rank matrix recovery methods in the literature we characterize the performance limits of our framework in terms of the tightness of the concentration property via novel bounds on the convergence rate and on the signal to noise ratio such that the theoretical guarantees are valid using the spectral initialization at the proper sample complexity

cluster ensembles a knowledge reuse framework for combining multiple partitions this paper introduces the problem of combining multiple partitionings of a set of objects into a single consolidated clustering without accessing the features or algorithms that determined these partitionings we first identify several application scenarios for the resultant knowledge reuse framework that we call cluster ensembles the cluster ensemble problem is then formalized as a combinatorial optimization problem in terms of shared mutual information in addition to a direct maximization approach we propose three effective and efficient techniques for obtaining high quality combiners consensus functions the first combiner induces a similarity measure from the partitionings and then reclusters the objects the second combiner is based on hypergraph partitioning the third one collapses groups of clusters into meta clusters which then compete for each object to determine the combined clustering due to the low computational costs of our techniques it is quite feasible to use a supra consensus function that evaluates all three approaches against the objective function and picks the best solution for a given situation we evaluate the effectiveness of cluster ensembles in three qualitatively different application scenarios i where the original clusters were formed based on non identical sets of features ii where the original clustering algorithms worked on non identical sets of objects and iii where a common data set is used and the main purpose of combining multiple clusterings is to improve the quality and robustness of the solution promising results are obtained in all three situations for synthetic as well as real data sets

discriminative graph embedding for label propagation in many applications the available information is encoded in graph structures this is a common problem in biological networks social networks web communities and document citations we investigate the problem of classifying nodes labels on a similarity graph given only a graph structure on the nodes conventional machine learning methods usually require data to reside in some euclidean spaces or to have a kernel representation applying these methods to nodes on graphs would require embedding the graphs into these spaces by embedding and then learning the nodes on graphs most methods are either flexible with different learning objectives or efficient enough for large scale applications we propose a method to embed a graph into a feature space for a discriminative purpose our idea is to include label information into the embedding process making the space representation tailored to the task we design embedding objective functions that the following learning formulations become spectral transforms we then reformulate these spectral transforms into multiple kernel learning problems our method while being tailored to the discriminative tasks is efficient and can scale to massive data sets we show the need of discriminative embedding on some simulations applying to biological network problems our method is shown to outperform baselines

risk convergence of centered kernel ridge regression with large dimensional data this paper carries out a large dimensional analysis of a variation of kernel ridge regression that we call centered kernel ridge regression ckrr also known in the literature as kernel ridge regression with offset this modified technique is obtained by accounting for the bias in the regression problem resulting in the old kernel ridge regression but with centered kernels the analysis is carried out under the assumption that the data is drawn from a gaussian distribution and heavily relies on tools from random matrix theory rmt under the regime in which the data dimension and the training size grow infinitely large with fixed ratio and under some mild assumptions controlling the data statistics we show that both the empirical and the prediction risks converge to a deterministic quantities that describe in closed form fashion the performance of ckrr in terms of the data statistics and dimensions inspired by this theoretical result we subsequently build a consistent estimator of the prediction risk based on the training data which allows to optimally tune the design parameters a key insight of the proposed analysis is the fact that asymptotically a large class of kernels achieve the same minimum prediction risk this insight is validated with both synthetic and real data

optimizing the gaussian kernel function with the formulated kernel target alignment criterion for two class pattern classification nowadays most of the current kernel learning approaches are showing good results in small datasets and fail to scale to large ones as such it is necessary to develop faster kernel optimization algorithms that perform better with larger datasets especially for the big data applications this paper presents a novel fast method to optimize the gaussian kernel function for two class pattern classification tasks where it is desirable for the kernel machines to use an optimized kernel that adapts well to the input data and the learning tasks we propose to optimize the gaussian kernel function by using the formulated kernel target alignment criterion by adopting the euler maclaurin formula and the local and global extremal properties of the approximate kernel separability criterion the approximate criterion function can be proved to have a determined global minimum point thus when the approximate criterion function is a sufficient approximation of the criterion function through using a newton based algorithm the proposed optimization is simply solved without being repeated the searching procedure with different starting points to locate the best local minimum the proposed method is evaluated on thirteen data sets with three gaussian kernel based learning algorithms the experimental results show that the criterion function has the determined global minimum point for the all thirteen data sets the proposed method achieves the best high time efficiency performance and the best overall classification performance

phaselift exact and stable signal recovery from magnitude measurements via convex programming suppose we wish to recover a signal input amssym font abc cmmib10 def bi 1 hbox abc 1 bi x in bbb c n from m intensity measurements of the form that is from data in which phase information is missing we prove that if the vectors are sampled independently and uniformly at random on the unit sphere then the signal x can be recovered exactly up to a global phase factor by solving a convenient semidefinite program a trace norm minimization problem this holds with large probability provided that m is on the order of and without any assumption about the signal whatsoever this novel result demonstrates that in some instances the combinatorial phase retrieval problem can be solved by convex programming techniques finally we also prove that our methodology is robust vis a vis additive noise 2012 wiley periodicals inc

multiple kernel learning algorithms in recent years several methods have been proposed to combine multiple kernels instead of using a single one these different kernels may correspond to using different notions of similarity or may be using information coming from multiple sources different representations or different feature subsets in trying to organize and highlight the similarities and differences between them we give a taxonomy of and review several multiple kernel learning algorithms we perform experiments on real data sets for better illustration and comparison of existing algorithms we see that though there may not be large differences in terms of accuracy there is difference between them in complexity as given by the number of stored support vectors the sparsity of the solution as given by the number of used kernels and training time complexity we see that overall using multiple kernels instead of a single one is useful and believe that combining kernels in a nonlinear or data dependent way seems more promising than linear combination in fusing information provided by simple linear kernels whereas linear methods are more reasonable when combining complex gaussian kernels