rough support vector regression abstract this paper describes the relationship between support vector regression svr and rough or interval patterns svr is the prediction component of the support vector techniques rough patterns are based on the notion of rough values which consist of upper and lower bounds and are used to effectively represent a range of variable values predictions of rough values in a variety of different forms within the context of interval algebra and fuzzy theory are attracting research interest an extension of svr called rough support vector regression rsvr is proposed to improve the modeling of rough patterns in particular it is argued that the upper and lower bounds should be modeled separately the proposal is shown to be a more flexible version of lower possibilistic regression model using insensitivity experimental results on the dow jones industrial average demonstrate the suggested rsvr modeling technique

duality for a class of nondifferentiable mathematical programming problems abstract two dual problems are proposed for the minimax problem minimize maxy y x y subject to g x 0 a duality theorem is established for each dual problem it is revealed that these problems are intimately related to a class of nondifferentiable programming problems

support vector data descriptions and k means clustering one class we present clustersvdd a methodology that unifies support vector data descriptions svdds and k means clustering into a single formulation this allows both methods to benefit from one another i e by adding flexibility using multiple spheres for svdds and increasing anomaly resistance and flexibility through kernels to k means in particular our approach leads to a new interpretation of k means as a regularized mode seeking algorithm the unifying formulation further allows for deriving new algorithms by transferring knowledge from one class learning settings to clustering settings and vice versa as a showcase we derive a clustering method for structured data based on a one class learning scenario additionally our formulation can be solved via a particularly simple optimization scheme we evaluate our approach empirically to highlight some of the proposed benefits on artificially generated data as well as on real world problems and provide a python software package comprising various implementations of primal and dual svdd as well as our proposed clustersvdd

kernel fuzzy c means with automatic variable weighting this paper presents variable wise kernel fuzzy c means clustering methods in which dissimilarity measures are obtained as sums of euclidean distances between patterns and centroids computed individually for each variable by means of kernel functions the advantage of the proposed approach over the conventional kernel clustering methods is that it allows us to use adaptive distances which change at each algorithm iteration and can either be the same for all clusters or different from one cluster to another this kind of dissimilarity measure is suitable to learn the weights of the variables during the clustering process improving the performance of the algorithms another advantage of this approach is that it allows the introduction of various fuzzy partition and cluster interpretation tools experiments with synthetic and benchmark datasets show the usefulness of the proposed algorithms and the merit of the fuzzy partition and cluster interpretation tools

spectral embedded clustering a framework for in sample and out of sample spectral clustering spectral clustering sc methods have been successfully applied to many real world applications the success of these sc methods is largely based on the manifold assumption namely that two nearby data points in the high density region of a low dimensional data manifold have the same cluster label however such an assumption might not always hold on high dimensional data when the data do not exhibit a clear low dimensional manifold structure e g high dimensional and sparse data the clustering performance of sc will be degraded and become even worse than k means clustering in this paper motivated by the observation that the true cluster assignment matrix for high dimensional data can be always embedded in a linear space spanned by the data we propose the spectral embedded clustering sec framework in which a linearity regularization is explicitly added into the objective function of sc methods more importantly the proposed sec framework can naturally deal with out of sample data we also present a new laplacian matrix constructed from a local regression of each pattern and incorporate it into our sec framework to capture both local and global discriminative information for clustering comprehensive experiments on eight real world high dimensional datasets demonstrate the effectiveness and advantages of our sec framework over existing sc methods and k means based clustering methods our sec framework significantly outperforms sc using the nystrom algorithm on unseen data

learning representative exemplars using one class gaussian process regression abstract an exemplar is an observation that represents a group of similar observations exemplars from data are examined to divide entire heterogeneous data into several homogeneous subgroups wherein each subgroup is represented by an exemplar with its inherent sparsity an exemplar based learning model provides a parsimonious model to represent or cluster large scale data a novel exemplar learning method using one class gaussian process gp regression is proposed in this study the proposed method constructs data distribution support from one class gp regression using automatic relevance determination prior and heterogeneous gp noise exemplars that correspond to the basis vectors of the constructed support function are then automatically located during the training process the proposed method is applied to various data sets to examine its operability characteristics of data representation and cluster analysis the exemplars of some real data generated by the proposed method are also reported

kernel discriminant transformation for image set based face recognition this study presents a novel kernel discriminant transformation kdt algorithm for face recognition based on image sets as each image set is represented by a kernel subspace we formulate a kdt matrix that maximizes the similarities of within kernel subspaces and simultaneously minimizes those of between kernel subspaces although the kdt matrix cannot be computed explicitly in a high dimensional feature space we propose an iterative kernel discriminant transformation algorithm to solve the matrix in an implicit way another perspective of similarity measure namely canonical difference is also addressed for matching each pair of the kernel subspaces and employed to simplify the formulation the proposed face recognition system is demonstrated to outperform existing still image based as well as image set based face recognition methods using the yale face database b labeled faces in the wild and a self compiled database

semi supervised clustering via multi level random walk a key issue of semi supervised clustering is how to utilize the limited but informative pairwise constraints in this paper we propose a new graph based constrained clustering algorithm named scrawl it is composed of two random walks with different granularities in the lower level random walk scrawl partitions the vertices i e data points into constrained and unconstrained ones according to whether they are in the pairwise constraints for every constrained vertex its influence range or the degrees of influence it exerts on the unconstrained vertices is encapsulated in an intermediate structure called component the edge set between each pair of components determines the affecting scope of the pairwise constraints in the higher level random walk scrawl enforces the pairwise constraints on the components so that the constraint influence can be propagated to the unconstrained edges at last we combine the cluster membership of all the components to obtain the cluster assignment for each vertex the promising experimental results on both synthetic and real world data sets demonstrate the effectiveness of our method highlightswe propose a new semi supervised clustering algorithm given pairwise constraints it can expand the influence of the pairwise constraints locally and proportionally it utilizes the pairwise constraints in an edge vertex edge manner it introduces an intermediate structure to uncover the underlying sub structures given a k nn sparse similarity matrix its time complexity is approximately linear

multi local model image set matching based on domain description image set matching attracted increasing attention in the field of pattern recognition recently there are a number of effective image set based matching methods under controlled environment however in the more complex environment like multi view and illumination changed it is still a challenging problem to develop unsupervised image set matching method to handle multi local model data to solve this problem in this paper we present a novel multi local model image set matching method based on data description techniques first every image set is divided into multi local models and each local model corresponds to a data domain that is we innovatively train a support vector data domain to describe each local model by means of the excellent data description ability of support vector data domain hence each image set can be expressed by a plurality of support vector data domain second a new similarity measure based on domain domain distance is proposed and then the distance between two image sets is converted to integrate the distance between pair wise domains finally the proposed method is evaluated on both set based face recognition and object classification tasks extensive experimental results show that the proposed method outperforms other state of the art set based matching methods in three public video databases highlightswe propose a novel multi local mode image set matching algorithm it is the first attempt to introduce the svdd to image set matching we solve the set matching as an optimization problem via domain description we formulate the image set matching as the distances between pair wise domains experimental results demonstrate the superior performance of our approach

data clustering using a model granular magnet we present a new approach to clustering based on the physical properties of an inhomogeneous ferromagnet no assumption is made regarding the underlying distribution of the data we assign a potts spin to each data point and introduce an interaction between neighboring points whose strength is a decreasing function of the distance between the neighbors this magnetic system exhibits three phases at very low temperatures it is completely ordered all spins are aligned at very high temperatures the system does not exhibit any ordering and in an intermediate regime clusters of relatively strongly coupled spins become ordered whereas different clusters remain uncorrelated this intermediate phase is identified by a jump in the order parameters the spin spin correlation function is used to partition the spins and the corresponding data points into clusters we demonstrate on three synthetic and three real data sets how the method works detailed comparison to the performance of other techniques clearly indicates the relative success of our method

a just nonsolvable torsion free group defined on the binary tree abstract a two generator torsion free subgroup of the group of finite state automorphisms of the binary tree is constructed having the properties of being just nonsolvable and residually torsion free solvable a presentation is produced for this subgroup in two generators and two relations together with their images under the iterated application of a certain simple substitution

soft clustering using weighted one class support vector machines this paper describes a new soft clustering algorithm in which each cluster is modelled by a one class support vector machine oc svm the proposed algorithm extends a previously proposed hard clustering algorithm also based on oc svm representation of clusters the key building block of our method is the weighted oc svm woc svm a novel tool introduced in this paper based on which an expectation maximization type soft clustering algorithm is defined a deterministic annealing version of the algorithm is also introduced and shown to improve the robustness with respect to initialization experimental results show that the proposed soft clustering algorithm outperforms its hard clustering counterpart namely in terms of robustness with respect to initialization as well as several other state of the art methods

unsupervised learning by minimal entropy encoding following basic principles of information theoretic learning in this paper we propose a novel approach to data clustering referred to as minimal entropy encoding mee which is based on a set of functions features projecting each input onto a minimum entropy configuration code inspired by traditional parsimony principles we seek solutions in reproducing kernel hilbert spaces and then we prove that the encoding functions are expressed in terms of kernel expansion in order to avoid trivial solutions the developed features must be as different as possible by means of a soft constraint on the empirical estimation of the entropy associated with the encoding functions this leads to an unconstrained optimization problem that can be efficiently solved by conjugate gradient we also investigate an optimization strategy based on concave convex algorithms the relationships with maximum margin clustering are studied showing that mee overcomes some of its critical issues such as the lack of a multiclass extension and the need to face problems with a large number of constraints a massive evaluation on several benchmarks of the proposed approach shows improvements over state of the art techniques both in terms of accuracy and computational complexity

optimality conditions for a nondifferentiable minimax programming in complex spaces abstract in this paper we study a nondifferentiable minimax programming in complex spaces we establish the kuhn tucker type necessary optimality conditions and prove the existence theorem for optimality in complex programming under generalized convexity

a novel clustering method based on hybrid k nearest neighbor graph abstract most of the existing clustering methods have difficulty in processing complex nonlinear data sets to remedy this deficiency in this paper a novel data model termed hybrid k nearest neighbor hknn graph which combines the advantages of mutual k nearest neighbor graph and k nearest neighbor graph is proposed to represent the nonlinear data sets moreover a clustering method based on the hknn graph chknn is proposed the chknn first generates several tight and small subclusters then merges these subclusters by exploiting the connectivity among them in order to select the optimal parameters for chknn we further propose an internal validity index termed k nearest neighbor index knni which can also be used to evaluate the validity of nonlinear clustering results by varying a control parameter experimental results on synthetic and real world data sets as well as that on the video clustering have demonstrated the significant improvement on performance over existing nonlinear clustering methods and internal validity indices

unsupervised image matching based on manifold alignment this paper challenges the issue of automatic matching between two image sets with similar intrinsic structures and different appearances especially when there is no prior correspondence an unsupervised manifold alignment framework is proposed to establish correspondence between data sets by a mapping function in the mutual embedding space we introduce a local similarity metric based on parameterized distance curves to represent the connection of one point with the rest of the manifold a small set of valid feature pairs can be found without manual interactions by matching the distance curve of one manifold with the curve cluster of the other manifold to avoid potential confusions in image matching we propose an extended affine transformation to solve the nonrigid alignment in the embedding space the comparatively tight alignments and the structure preservation can be obtained simultaneously the point pairs with the minimum distance after alignment are viewed as the matchings we apply manifold alignment to image set matching problems the correspondence between image sets of different poses illuminations and identities can be established effectively by our approach

applying the possibilistic c means algorithm in kernel induced spaces in this paper we study a kernel extension of the classic possibilistic c means in the proposed extension we implicitly map input patterns into a possibly high dimensional space by means of positive semidefinite kernels in this new space we model the mapped data by means of the possibilistic clustering algorithm we study in more detail the special case where we model the mapped data using a single cluster only since it turns out to have many interesting properties the modeled memberships in kernel induced spaces yield a modeling of generic shapes in the input space we analyze in detail the connections to one class support vector machines and kernel density estimation thus suggesting that the proposed algorithm can be used in many scenarios of unsupervised learning in the experimental part we analyze the stability and the accuracy of the proposed algorithm on some synthetic and real datasets the results show high stability and good performances in terms of accuracy

a survey of kernel and spectral methods for clustering clustering algorithms are a useful tool to explore data structures and have been employed in many disciplines the focus of this paper is the partitioning clustering problem with a special interest in two recent approaches kernel and spectral methods the aim of this paper is to present a survey of kernel and spectral clustering methods two approaches able to produce nonlinear separating hypersurfaces between clusters the presented kernel clustering methods are the kernel version of many classical clustering algorithms e g k means som and neural gas spectral clustering arise from concepts in spectral graph theory and the clustering problem is configured as a graph cut problem where an appropriate objective function has to be optimized an explicit proof of the fact that these two paradigms have the same objective is reported since it has been proven that these two seemingly different approaches have the same mathematical foundation besides fuzzy kernel clustering methods are presented as extensions of kernel k means clustering algorithm

support vector clustering we present a novel clustering method using the approach of support vector machines data points are mapped by means of a gaussian kernel to a high dimensional feature space where we search for the minimal enclosing sphere this sphere when mapped back to data space can separate into several components each enclosing a separate cluster of points we present a simple algorithm for identifying these clusters the width of the gaussian kernel controls the scale at which the data is probed while the soft margin constant helps coping with outliers and overlapping clusters the structure of a dataset is explored by varying the two parameters maintaining a minimal number of support vectors to assure smooth cluster boundaries we demonstrate the performance of our algorithm on several datasets

incorporating the loss function into discriminative clustering of structured outputs clustering using the hilbert schmidt independence criterion cluhsic is a recent clustering algorithm that maximizes the dependence between cluster labels and data observations according to the hilbert schmidt independence criterion hsic it is unique in that structure information on the cluster outputs can be easily utilized in the clustering process however while the choice of the loss function is known to be very important in supervised learning with structured outputs we will show in this paper that cluhsic is implicitly using the often inappropriate zero one loss we propose an extension called cluhsical which stands for clustering using hsic and loss which explicitly considers both the output dependency and loss function its optimization problem has the same form as cluhsic except that its partition matrix is constructed in a different manner experimental results on a number of datasets with structured outputs show that cluhsical often outperforms cluhsic in terms of both structured loss and clustering accuracy

face recognition using sparse approximated nearest points between image sets we propose an efficient and robust solution for image set classification a joint representation of an image set is proposed which includes the image samples of the set and their affine hull model the model accounts for unseen appearances in the form of affine combinations of sample images to calculate the between set distance we introduce the sparse approximated nearest point sanp sanps are the nearest points of two image sets such that each point can be sparsely approximated by the image samples of its respective set this novel sparse formulation enforces sparsity on the sample coefficients and jointly optimizes the nearest points as well as their sparse approximations unlike standard sparse coding the data to be sparsely approximated are not fixed a convex formulation is proposed to find the optimal sanps between two sets and the accelerated proximal gradient method is adapted to efficiently solve this optimization we also derive the kernel extension of the sanp and propose an algorithm for dynamically tuning the rbf kernel parameter while matching each pair of image sets comprehensive experiments on the ucsd honda cmu mobo and youtube celebrities face datasets show that our method consistently outperforms the state of the art

spectralcat categorical spectral clustering of numerical and nominal data data clustering is a common technique for data analysis which is used in many fields including machine learning data mining customer segmentation trend analysis pattern recognition and image analysis although many clustering algorithms have been proposed most of them deal with clustering of one data type numerical or nominal or with mix data type numerical and nominal and only few of them provide a generic method that clusters all types of data it is required for most real world applications data to handle both feature types and their mix in this paper we propose an automated technique called spectralcat for unsupervised clustering of high dimensional data that contains numerical or nominal or mix of attributes we suggest to automatically transform the high dimensional input data into categorical values this is done by discovering the optimal transformation according to the calinski harabasz index for each feature and attribute in the dataset then a method for spectral clustering via dimensionality reduction of the transformed data is applied this is achieved by automatic non linear transformations which identify geometric patterns in the data and find the connections among them while projecting them onto low dimensional spaces we compare our method to several clustering algorithms using 16 public datasets from different domains and types the experiments demonstrate that our method outperforms in most cases these algorithms

optimality and duality for minmax problems involving arcwise connected and generalized arcwise connected functions abstract in this paper we establish necessary optimality conditions for a static minmax programming problem of the form min max y y x y subject tog x 0 in terms of the right derivatives of the functions with respect to the same arc various theorems giving sufficient optimality conditions are proved a mond weir type dual is proposed and duality results are established under arcwise connectedness and generalized arcwise connectedness assumptions

finknn a fuzzy interval number k nearest neighbor classifier for prediction of sugar production from populations of samples this work introduces finknn a k nearest neighbor classifier operating over the metric lattice of conventional interval supported convex fuzzy sets we show that for problems involving populations of measurements data can be represented by fuzzy interval numbers fins and we present an algorithm for constructing fins from such populations we then present a lattice theoretic metric distance between fins with arbitrary shaped membership functions which forms the basis for finknn s similarity measurements we apply finknn to the task of predicting annual sugar production based on populations of measurements supplied by hellenic sugar industry we show that finknn improves prediction accuracy on this task and discuss the broader scope and potential utility of these techniques

robust clustering using outlier sparsity regularization notwithstanding the popularity of conventional clustering algorithms such as k means and probabilistic clustering their clustering results are sensitive to the presence of outliers in the data even a few outliers can compromise the ability of these algorithms to identify meaningful hidden structures rendering their outcome unreliable this paper develops robust clustering algorithms that not only aim to cluster the data but also to identify the outliers the novel approaches rely on the infrequent presence of outliers in the data which translates to sparsity in a judiciously chosen domain leveraging sparsity in the outlier domain outlier aware robust k means and probabilistic clustering approaches are proposed their novelty lies on identifying outliers while effecting sparsity in the outlier domain through carefully chosen regularization a block coordinate descent approach is developed to obtain iterative algorithms with convergence guarantees and small excess computational complexity with respect to their non robust counterparts kernelized versions of the robust clustering algorithms are also developed to efficiently handle high dimensional data identify nonlinearly separable clusters or even cluster objects that are not represented by vectors numerical tests on both synthetic and real datasets validate the performance and applicability of the novel algorithms

maximal linear embedding for dimensionality reduction over the past few decades dimensionality reduction has been widely exploited in computer vision and pattern analysis this paper proposes a simple but effective nonlinear dimensionality reduction algorithm named maximal linear embedding mle mle learns a parametric mapping to recover a single global low dimensional coordinate space and yields an isometric embedding for the manifold inspired by geometric intuition we introduce a reasonable definition of locally linear patch maximal linear patch mlp which seeks to maximize the local neighborhood in which linearity holds the input data are first decomposed into a collection of local linear models each depicting an mlp these local models are then aligned into a global coordinate space which is achieved by applying mds to some randomly selected landmarks the proposed alignment method called landmarks based global alignment lga can efficiently produce a closed form solution with no risk of local optima it just involves some small scale eigenvalue problems while most previous aligning techniques employ time consuming iterative optimization compared with traditional methods such as isomap and lle our mle yields an explicit modeling of the intrinsic variation modes of the observation data extensive experiments on both synthetic and real data indicate the effectivity and efficiency of the proposed algorithm

clustering irregular shapes using high order neurons this article introduces a method for clustering irregularly shaped data arrangements using high order neurons complex analytical shapes are modeled by replacing the classic synaptic weight of the neuron by high order tensors in homogeneous coordinates in the first and second order cases this neuron corresponds to a classic neuron and to an ellipsoidalmetric neuron we show how high order shapes can be formulated to follow the maximum correlation activation principle and permit simple local hebbian learning we also demonstrate decomposition of spatial arrangements of data clusters including very close and partially overlapping clusters which are difficult to distinguish using classic neurons superior results are obtained for the iris data

statistical computations on grassmann and stiefel manifolds for image and video based recognition in this paper we examine image and video based recognition applications where the underlying models have a special structure the linear subspace structure we discuss how commonly used parametric models for videos and image sets can be described using the unified framework of grassmann and stiefel manifolds we first show that the parameters of linear dynamic models are finite dimensional linear subspaces of appropriate dimensions unordered image sets as samples from a finite dimensional linear subspace naturally fall under this framework we show that an inference over subspaces can be naturally cast as an inference problem on the grassmann manifold to perform recognition using subspace based models we need tools from the riemannian geometry of the grassmann manifold this involves a study of the geometric properties of the space appropriate definitions of riemannian metrics and definition of geodesics further we derive statistical modeling of inter and intraclass variations that respect the geometry of the space we apply techniques such as intrinsic and extrinsic statistics to enable maximum likelihood classification we also provide algorithms for unsupervised clustering derived from the geometry of the manifold finally we demonstrate the improved performance of these methods in a wide variety of vision applications such as activity recognition video based face recognition object recognition from image sets and activity based video clustering

boosted manifold principal angles for image set based recognition in this paper we address the problem of classifying vector sets we motivate and introduce a novel method based on comparisons between corresponding vector subspaces in particular there are two main areas of novelty i we extend the concept of principal angles between linear subspaces to manifolds with arbitrary nonlinearities ii it is demonstrated how boosting can be used for application optimal principal angle fusion the strengths of the proposed method are empirically demonstrated on the task of automatic face recognition afr in which it is shown to outperform state of the art methods in the literature

an extension to fuzzy cognitive maps for classification and prediction fuzzy cognitive maps fcms as an illustrative causative representation of modeling and manipulation of complex systems can be used to model the dynamic behavior of the investigated systems however due to defects in expression and architecture the traditional fcms and most of their relevant extensions are not applicable to classification problems to solve this problem this paper presents an approach that directly extends the model by translating the reasoning mechanism of traditional fcms to a set of fuzzy if then rules moreover the proposed approach fully considers the contribution of the inputs to the activation of the fuzzy rules and quantifies the causalities using mutual subsethood which works in conjunction with volume defuzziflcation in a gradient descent learning framework in this manner our approach enhances the capability of the conventional fcms to automatically identify membership functions and quantify causalities despite the increase in the number of tunable parameters experimental results show that the proposed approach efficiently extends the application of the traditional fcms into classification problems while keeping the ability for prediction and approximation

optimality and duality for complex nondifferentiable fractional programming abstract both parametric and nonparametric necessary and sufficient optimality conditions are established for a class of complex nondifferentiable fractional programming problems containing generalized convex functions subsequently these optimality criteria are utilized as a basis for constructing one parametric and two other parameter free dual models with appropriate duality theorems

parametric duality on minimax programming involving generalized convexity in complex space in this paper we employ generalized convexity of complex functions to establish several sufficient optimality conditions for minimax programming in complex spaces using such criteria we constitute a parametrical dual and establish the weak strong and strict converse duality theorems in the framework

an examination of procedures for determining the number of clusters in a data set a monte carlo evaluation of 30 procedures for determining the number of clusters was conducted on artificial data sets which contained either 2 3 4 or 5 distinct nonoverlapping clusters to provide a variety of clustering solutions the data sets were analyzed by four hierarchical clustering methods external criterion measures indicated excellent recovery of the true cluster structure by the methods at the correct hierarchy level thus the clustering present in the data was quite strong the simulation results for the stopping rules revealed a wide range in their ability to determine the correct number of clusters in the data several procedures worked fairly well whereas others performed rather poorly thus the latter group of rules would appear to have little validity particularly for data sets containing distinct clusters applied researchers are urged to select one or more of the better criteria however users are cautioned that the performance of some of the criteria may be data dependent

ellipsoidal support vector clustering for functional mri analysis as an exploratory approach the clustering of fmri time series has proved its effectiveness in analyzing the functional mri especially in the detection of activated regions due to the arbitrary distribution of fmri time series in the temporal domain imposing simple assumption on the data structure usually could be misleading and limit the detector s performance therefore a true data driven clustering algorithm that adapts to the data structure is preferred and only high level control over the clustering procedure is desired support vector clustering svc is a suitable one in some extent because of its advantages such as no cluster shape restriction no need to explicitly specify the number of clusters and the mechanism in outlier elimination in this work we propose an extension of the svc to step further toward a data sensitive detector this approach is named as ellipsoidal support vector clustering esvc to be robust to noise the clustering is performed on features extracted from the fmri time series via fourier transform experimental results on simulated and real data sets demonstrate the effectiveness of incorporating data structure in clustering fmri time series

dynamic cluster formation using level set methods density based clustering has the advantages for 1 allowing arbitrary shape of cluster and 2 not requiring the number of clusters as input however when clusters touch each other both the cluster centers and cluster boundaries as the peaks and valleys of the density distribution become fuzzy and difficult to determine we introduce the notion of cluster intensity function cif which captures the important characteristics of clusters when clusters are well separated cifs are similar to density functions but when clusters become closed to each other cifs still clearly reveal cluster centers cluster boundaries and degree of membership of each data point to the cluster that it belongs clustering through bump hunting and valley seeking based on these functions are more robust than that based on density functions obtained by kernel density estimation which are often oscillatory or oversmoothed these problems of kernel density estimation are resolved using level set methods and related techniques comparisons with two existing density based methods valley seeking and dbscan are presented which illustrate the advantages of our approach

smoothly approximated support vector domain description support vector domain description svdd is a well known tool for pattern analysis when only positive examples are reliable the svdd model is often fitted by solving a quadratic programming problem which is time consuming this paper attempts to fit svdd in the primal form directly however the primal objective function of svdd is not differentiable which prevents the well behaved gradient based optimization methods from being applicable as such we propose to approximate the primal objective function of svdd by a differentiable function and a conjugate gradient method is applied to minimize the smoothly approximated objective function extensive experiments on pattern classification were conducted and compared to the quadratic programming based svdd the proposed approach is much more computationally efficient and yields similar classification performance on these problems highlightssmoothly approximates the objective function of support vector domain description gradient based optimization method could be applied to the smoothed model the proposed algorithms have asymptotic training complexity n 2 the algorithm is easy to implement without requiring any optimization package

on support vector regression machines with linguistic interpretation of the kernel matrix initially the idea of approximate reasoning using generalized modus ponens and a fuzzy implication is recalled next a fuzzy system based on logical interpretation of if then rules and with parametric conclusions is presented then it is shown that global and local e insensitive learning of the above fuzzy system may be presented as the learning of a support vector regression machine with a special type of a kernel matrix obtained from clustering the kernel matrix may be interpreted in terms of linguistic values based on the premises of if then rules a new method of obtaining a fuzzy system by means of a support vector machine svm with a data dependent kernel matrix is introduced this paper contains examples of a svm used to design fuzzy models of real life data simulation results show an improvement in the generalization ability of a fuzzy system learned by the new method compared with traditional learning methods

a novel kernel method for clustering kernel methods are algorithms that by replacing the inner product with an appropriate positive definite function implicitly perform a nonlinear mapping of the input data into a high dimensional feature space in this paper we present a kernel method for clustering inspired by the classical k means algorithm in which each cluster is iteratively refined using a one class support vector machine our method which can be easily implemented compares favorably with respect to popular clustering algorithms like k means neural gas and self organizing maps on a synthetic data set and three uci real data benchmarks iris data wisconsin breast cancer database spam database

optimality conditions for nondifferentiable minimax fractional programming with complex variables we show that a minimax fractional programming problem is equivalent to a minimax nonfractional parametric problem for a given parameter in complex space we establish the necessary and sufficient optimality conditions of nondifferentiable minimax fractional programming problem with complex variables under generalized convexities

estimating the support of a high dimensional distribution suppose you are given some data set drawn from an underlying probability distribution p and you want to estimate a simple subset s of input space such that the probability that a test point drawn from p lies outside of s equals some a priori specified value between 0 and 1 we propose a method to approach this problem by trying to estimate a function f that is positive on s and negative on the complement the functional form of f is given by a kernel expansion in terms of a potentially small subset of the training data it is regularized by controlling the length of the weight vector in an associated feature space the expansion coefficients are found by solving a quadratic programming problem which we do by carrying out sequential optimization over pairs of input patterns we also provide a theoretical analysis of the statistical performance of our algorithm the algorithm is a natural extension of the support vector algorithm to the case of unlabeled data

duality for a class of nondifferentiable mathematical programming problems in complex space craven and mond 1 have given fritz john type necessary conditions for a class of nonlinear programming problems in complex space over polyhedral cones here we derive the necessary and sufficient conditions for the static minimax problems in complex space which are extensions of the corresponding real space conditions of 4 these conditions are then used to extend some duality results of 2 for a general class of nondifferentiable programming problems to complex space over arbitrary polyhedral cones the complex minimax problem that we consider seeks to choose e so e c g c e s which minimizes

duality for nondifferentiable minimax programming in complex spaces abstract employing the optimality necessary and sufficient conditions of a nondifferentiable minimax programming problem in complex spaces we formulate a one parametric dual and a parameter free dual problems on both dual problems we establish three duality theorems weak strong and strict converse duality theorem and prove that there is no duality gap between the two dual problems with respect to the primal problem under some generalized convexities of complex functions in the complex programming problem

online primal dual learning for a data dependent multi kernel combination model with multiclass visual categorization applications the task of visual categorization becomes very challenging as the number of samples or classes increases this is mainly due to large memory requirements or a crowded semantic space which causes class separation difficulties in this paper a new approach called oprdum2 online primal dual multiple kernel multiple class is proposed this new algorithm can significantly reduce the complexity of visual categorization under this approach an elegant heterogeneous feature fusion machine is used to consolidate complementary class discriminative information and online learning is applied to deal with large sample sizes to be specific a data dependent multi kernel machine that fuses multiple heterogeneous features in a nonlinear manner is defined by extending the multi kernel learning model and multivariate hinge loss and conservative updating rules are used to increase the sample level sparsity in the model given the intrinsic hierarchical structure of samples features and classes an innovative triple mixed norm regularizer with strong convexity is utilized to facilitate optimization an iterative solution is derived using the language of duality leading to a low regret bound of o t where t is the number of online iterations this paper details a proof of the algorithm s convergence and describes extensive visual data classification experiments that demonstrate the effectiveness and robustness of this new approach

position regularized support vector domain description support vector domain description svdd is an effective method for describing a set of objects as a basic tool several application oriented extensions have been developed such as support vector clustering svc svdd based k means svddk means and support vector based algorithm for clustering data streams svstream despite its significant success one inherent drawback is that the description is very sensitive to the selection of the trade off parameter which is hard to estimate in practice and affects the extensive approaches significantly to tackle this problem we propose a novel position regularized support vector domain description psvdd in the proposed psvdd the complexity of the sphere surface is adaptively regularized by assigning a position based weighting to each data point which is computed according to the distance between the corresponding feature space image and the mean of feature space images to demonstrate the effectiveness of the proposed psvdd we apply the position based weighting to improve two important clustering extensions i e svc and svddk means which respectively result in two new clustering approaches termed psvc and psvddk means experimental results on several real world data sets validate the significant improvement achieved by psvc and psvddk means

kernel based hard clustering methods with kernelization of the metric and automatic weighting of the variables this paper presents kernel based hard clustering methods with kernelization of the metric and automatic weighting of the variables the proposed methodology is supported by the fact that a kernel function can be written as a sum of kernels evaluated separately on each variable thus in the proposed algorithms dissimilarity measures are obtained as sums of euclidean distances between patterns and centroids computed individually for each variable by means of kernels the main advantage of this approach over the conventional approach is that it allows the use of kernelized adaptive distances which are suitable to learn the weights of the variables dynamically improving the performance of the algorithms moreover various partition and cluster interpretation tools are introduced experiments with a number of benchmark datasets corroborate the usefulness of the proposed algorithms and the merit of the partition and cluster interpretation tools highlightspresents kernel based clustering methods with automatic weighting of the variables kernelized local and global adaptive distances are introduced the proposed algorithms are suitable to learn the weights of the variables partition and cluster interpretation tools are given experiments with several benchmark data sets corroborate the proposed methods

wreath operations in the group of automorphisms of the binary tree abstract a new operation called tree wreathing is defined on groups of automorphisms of the binary tree given a countable residually finite 2 group h and a free abelian group k of finite rank r this operation produces uniformly copies of these as automorphism groups of the binary tree such that the group generated by them is an over group of the restricted wreath product h k indeed g contains a normal subgroup n which is an infinite direct sum of copies of the derived group h and the quotient group g n is isomorphic to h k the tree wreathing construction preserves the properties of solvability torsion freeness and of having finite state i e generated by finite automata a faithful representation of any free metabelian group of finite rank is obtained as a finite state group of automorphisms of the binary tree

on the automorphism group of the one rooted binary tree abstract let a be the automorphism group of the one rooted regular binary tree t 2 and g the subgroup of a consisting of those automorphisms admitting a finite description in their action on t 2 let n a g be the normaliser of g in a let aut g be the group of automorphisms of g and let end a g be the semigroup of endomorphisms of g induced by conjugation by elements of a then g is the infinite iterated wreath product c 2 c 2 and a is the topological limit of g we study in some detail the structure of g aut g and end a g in particular we prove n a g is isomorphic to aut g contains a copy of a itself and is a proper subgroup of end a g furthermore we discuss connections with automata and introduce the notion of functionally recursive automorphisms of t 2

an efficient algorithm for the smallest enclosing ball problem in high dimensions abstract consider the problem of computing the smallest enclosing ball of a set of m balls in r n this problem arises in many applications such as location analysis military operations and pattern recognition etc in this paper we reformulate this problem as an unconstrained convex optimization problem involving the maximum function max 0 t and then develop a simple algorithm particularly suitable for problems in high dimensions this algorithm could efficiently handle problems of dimension n up to 10 000 under a moderately large m as well as problems of dimension m up to 10 000 under a moderately large n numerical results are given to show the efficiency of algorithm