rough support vector regression abstract this paper describes the relationship between support vector regression svr and rough or interval patterns svr is the prediction component of the support vector techniques rough patterns are based on the notion of rough values which consist of upper and lower bounds and are used to effectively represent a range of variable values predictions of rough values in a variety of different forms within the context of interval algebra and fuzzy theory are attracting research interest an extension of svr called rough support vector regression rsvr is proposed to improve the modeling of rough patterns in particular it is argued that the upper and lower bounds should be modeled separately the proposal is shown to be a more flexible version of lower possibilistic regression model using insensitivity experimental results on the dow jones industrial average demonstrate the suggested rsvr modeling technique

clustering high dimensional data a graph based relaxed optimization approach there is no doubt that clustering is one of the most studied data mining tasks nevertheless it remains a challenging problem to solve despite the many proposed clustering approaches graph based approaches solve the clustering task as a global optimization problem while many other works are based on local methods in this paper we propose a novel graph based algorithm gbr that relaxes some well defined method even as improving the accuracy whilst keeping it simple the primary motivation of our relaxation of the objective is to allow the reformulated objective to find well distributed cluster indicators for complicated data instances this relaxation results in an analytical solution that avoids the approximated iterative methods that have been adopted in many other graph based approaches the experiments on synthetic and real data sets show that our relaxation accomplishes excellent clustering results our key contributions are 1 we provide an analytical solution to solve the global clustering task as opposed to approximated iterative approaches 2 a very simple implementation using existing optimization packages 3 an algorithm with relatively less computation time over the number of data instances to cluster than other well defined methods in the literature

support vector machine soft margin classifiers error analysis the purpose of this paper is to provide a pac error analysis for the q norm soft margin classifier a support vector machine classification algorithm it consists of two parts regularization error and sample error while many techniques are available for treating the sample error much less is known for the regularization error and the corresponding approximation error for reproducing kernel hilbert spaces we are mainly concerned about the regularization error it is estimated for general distributions by a k functional in weighted lq spaces for weakly separable distributions i e the margin may be zero satisfactory convergence rates are provided by means of separating functions a projection operator is introduced which leads to better sample error estimates especially for small complexity kernels the misclassification error is bounded by the v risk associated with a general class of loss functions v the difficulty of bounding the offset is overcome polynomial kernels and gaussian kernels are used to demonstrate the main results the choice of the regularization parameter plays an important role in our analysis

personalized mode transductive spanning svm classification tree personalized transductive learning ptl builds a unique local model for classification of individual test samples and is therefore practically neighborhood dependant i e a specific model is built in a subspace spanned by a set of samples adjacent to the test sample while existing ptl methods usually define the neighborhood by a predefined dis similarity measure this paper introduces a new concept of a knowledgeable neighborhood and a transductive support vector machine svm classification tree t svmt for ptl the neighborhood of a test sample is constructed over the classification knowledge modelled by regional svms and a set of such svms adjacent to the test sample is systematically aggregated into a t svmt compared to a regular svm and other svmts a t svmt by virtue of the aggregation of svms has an inherent superiority in classifying class imbalanced datasets the t svmt has also solved the over fitting problem of all previous svmts since it aggregates neighborhood knowledge and thus significantly reduces the size of the svm tree the properties of the t svmt are evaluated through experiments on a synthetic dataset eight bench mark cancer diagnosis datasets as well as a case study of face membership authentication

manifold regularization a geometric framework for learning from labeled and unlabeled examples we propose a family of learning algorithms based on a new form of regularization that allows us to exploit the geometry of the marginal distribution we focus on a semi supervised framework that incorporates labeled and unlabeled data in a general purpose learner some transductive graph learning algorithms and standard methods including support vector machines and regularized least squares can be obtained as special cases we use properties of reproducing kernel hilbert spaces to prove new representer theorems that provide theoretical basis for the algorithms as a result in contrast to purely graph based approaches we obtain a natural out of sample extension to novel examples and so are able to handle both transductive and truly semi supervised settings we present experimental evidence suggesting that our semi supervised algorithms are able to use unlabeled data effectively finally we have a brief discussion of unsupervised and fully supervised learning within our general framework

on the structure of generalized rough sets in this paper we consider some fundamental properties of generalized rough sets induced by binary relations on algebras and show that 1 any reflexive binary relation determines a topology 2 if q is a reflexive and symmetric relation on a set x then o email protected x q a a is a topology such that a is open if and only if it is closed 3 conversely for every topological space x o satisfying the condition that a is open if and only if it is closed there exists a reflexive and symmetric relation r such that o email protected x r a a 4 let q be an equivalence relation on x for any pseudo w closed subset a of x q a is an w closed set if and only if w x x x email protected a for any email protected x moreover we consider properties of generalized rough sets

the equivalence of strict convexity and injectivity of the gradient in bounded level sets it is shown that lipschitzian functions are strictly convex if and only if their generalized gradients are disjoint at distinct interior points of a given bounded level set

a comparative study of fuzzy sets and rough sets this paper reviews and compares theories of fuzzy sets and rough sets two approaches for the formulation of fuzzy sets are reviewed one is based on many valued logic and the other is based on modal logic two views of rough sets are presented set oriented view and operator oriented view rough sets under set oriented view are closely related to fuzzy sets which leads to non truth functional fuzzy set operators both of them may be considered as deviations of classical set algebra in contrast rough sets under operator oriented view are different from fuzzy sets and may be regarded as an extension of classical set algebra

the relationship between attribute reducts in rough sets and minimal vertex covers of graphs the problems to find attribute reduction in rough sets and to obtain the minimal vertex cover for graphs are both np hard problems this paper studies the relationship between the two problems the vertex cover problem for graphs from the perspective of rough sets is first investigated the attribute reduction of an information system is then studied in the framework of graph theory the results in this paper show that finding the minimal vertex cover of a graph is equivalent to finding the attribute reduction of an information system induced from the graph conversely the attribute reduction computation can be translated into the calculation of the minimal vertex cover of a derivative graph finally a new algorithm for the vertex cover problem based on rough sets is presented furthermore experiments are conducted to verify the effectiveness of the proposed method

relational interpretations of neighborhood operators and rough set approximation operators this paper presents a framework for the formulation interpretation and comparison of neighborhood systems and rough set approximations using the more familiar notion of binary relations a special class of neighborhood systems called 1 neighborhood systems is introduced three extensions of pawlak approximation operators are analyzed properties of neighborhood and approximation operators are studied and their connections are examined

the global convergence of partitioned bfgs on problems with convex decompositions and lipschitzian gradients the main purpose of this paper is the extension of powell s 1976 global convergence result to the partitioned bfgs method introduced by griewank and toint 1982 even in the unpartitioned case the original result is strengthened because the search directions need not be computed exactly and the gradient is only required to be lipschitzian rather than differentiable using the functional of byrd and nocedal 1987 a strong form of r superlinear convergence is obtained if the element functions are uniformly convex and their gradients are strictly differentiable at the minimizerx only in order to deal with the possibility of singular functions we utilize a damping of the bfgs update that becomes inactive if the problem turns out to be regular nearx

a short note on algebraic t rough sets our aim in this paper is to introduce the concept of a set valued homomorphism for groups which is a generalization of ordinary homomorphism then using the definitions of lower inverse and upper inverse we introduce the definition of uniform set valued homomorphism and prove that every set valued homomorphism is uniform finally the concept of probabilistic t rough set is defined

automatic image annotation by semi supervised manifold kernel density estimation the insufficiency of labeled training data is a major obstacle in automatic image annotation to tackle this problem we propose a semi supervised manifold kernel density estimation ssmkde approach based on a recently proposed manifold kde method our contributions are twofold first ssmkde leverages both labeled and unlabeled samples and formulates all data in a manifold structure which enables a more accurate label prediction second the relationship between kde based methods and graph based semi supervised learning ssl methods is analyzed which helps to better understand graph based ssl methods extensive experiments demonstrate the superiority of ssmkde over existing kde based and graph based ssl methods

on the effectiveness of laplacian normalization for graph semi supervised learning this paper investigates the effect of laplacian normalization in graph based semi supervised learning to this end we consider multi class transductive learning on graphs with laplacian regularization generalization bounds are derived using geometric properties of the graph specifically by introducing a definition of graph cut from learning theory we obtain generalization bounds that depend on the laplacian regularizer we then use this analysis to better understand the role of graph laplacian matrix normalization under assumptions that the cut is small we derive near optimal normalization factors by approximately minimizing the generalization bounds the analysis reveals the limitations of the standard degree based normalization method in that the resulting normalization factors can vary significantly within each connected component with the same class label which may cause inferior generalization performance our theory also suggests a remedy that does not suffer from this problem experiments confirm the superiority of the normalization scheme motivated by learning theory on artificial and real world data sets

semantic preserving distance metric learning and applications how do we accurately browse a large set of images or efficiently annotate the images from an image library image clustering methods are invaluable tools for applications such as content based image retrieval and image annotation to perform these tasks it is critical to have proper features to describe the visual and semantic content of images and to define an accurate distance metric to measure the dissimilarity between any two images however existing methods which adopt the features of color histograms edge direction histograms and shape context lack the ability to describe semantic content to solve this problem we propose a new approach that utilizes user provided pairwise constraints to describe the semantic relationship between two images a semantic preserving distance metric learning sp dml algorithm is developed to explore the complementary characteristics of the visual features and pairwise constraints in a unified feature space in this space the learned distance metric can be used to measure the dissimilarity between two images specifically the manifold structure adopted in sp dml is revealed by the image s visual features to integrate semantic contents in distance metric learning sp dml utilizes pairwise constraints to build semantic patches and align these patches to obtain the optimal distance metric for the new feature space experimental results in image clustering demonstrate that the performance of sp dml is appealing

semisupervised multicategory classification with imperfect model semisupervised learning has been of growing interest over the past years and many methods have been proposed while existing semisupervised methods have shown some promising empirical performances their development has been based largely on heuristics in this paper we investigate semisupervised multicategory classification with an imperfect mixture density model in the proposed model the training data come from a probability distribution which can be modeled imperfectly by an identifiable mixture distribution furthermore we propose a semisupervised multicategory classification method and establish its generalization error bounds the theoretical analysis illustrates that the proposed method can utilize unlabeled data effectively and can achieve fast convergence rate

error bounds of multi graph regularized semi supervised classification in this paper we investigate the generalization performance of the multi graph regularized semi supervised classification algorithm associated with the hinge loss we provide estimates for the excess misclassification error of multi graph regularized classifiers and show the relations between the generalization performance and the structural invariants of data graphs experiments performed on real database demonstrate the effectiveness of our theoretical analysis

bounds on error expectation for support vector machines we introduce the concept of span of support vectors sv and show that the generalization ability of support vector machines svm depends on this new geometrical concept we prove that the value of the span is always smaller and can be much smaller than the diameter of the smallest sphere containing the support vectors used in previous bounds vapnik 1998 we also demonstate experimentally that the prediction of the test error given by the span is very accurate and has direct application in model selection choice of the optimal parameters of the svm

some mathematical structures for computational information abstract this paper is about the basic underlying mathematical structures of various types of imprecise information set theory and probability measures are basic ingredients in extracting some specific type of information and in reasoning with uncertain knowledge but in this era of information technology it is desirable to consider also more complex types of information for example information coming from human perception see e g l a zadeh ieee trans circuits and systems 45 1 1999 105 119 as a first approximation the classical ingredients could be generalized to adequately model new types of information as such we will describe in this paper four types of non classical sets namely random sets rough sets conditional sets and fuzzy sets as well as their associated non additive measures

on rough set and fuzzy sublattice let l be a lattice with the least element 0 and the greatest element 1 and let be a full congruence relation on l in this paper the notion of upper and lower approximations of a fuzzy subset of l is introduced and some important properties will be studied

risk sensitive loss functions for sparse multi category classification problems in this paper we propose two risk sensitive loss functions to solve the multi category classification problems where the number of training samples is small and or there is a high imbalance in the number of samples per class such problems are common in the bio informatics medical diagnosis areas the most commonly used loss functions in the literature do not perform well in these problems as they minimize only the approximation error and neglect the estimation error due to imbalance in the training set the proposed risk sensitive loss functions minimize both the approximation and estimation error we present an error analysis for the risk sensitive loss functions along with other well known loss functions using a neural architecture classifiers incorporating these risk sensitive loss functions have been developed and their performance evaluated for two real world multi class classification problems viz a satellite image classification problem and a micro array gene expression based cancer classification problem to study the effectiveness of the proposed loss functions we have deliberately imbalanced the training samples in the satellite image problem and compared the performance of our neural classifiers with those developed using other well known loss functions the results indicate the superior performance of the neural classifier using the proposed loss functions both in terms of the overall and per class classification accuracy performance comparisons have also been carried out on a number of benchmark problems where the data is normal i e not sparse or imbalanced results indicate similar or better performance of the proposed loss functions compared to the well known loss functions

mining diagnostic rules from clinical databases using rough sets and medical diagnostic model since rule induction methods generate rules whose lengths are the shortest for discrimination between given classes they tend to generate rules too short for medical experts thus these rules are difficult for the experts to interpret from the viewpoint of domain knowledge in this paper the characteristics of experts rules are closely examined and a new approach to generate diagnostic rules is introduced the proposed method focuses on the hierarchical structure of differential diagnosis and consists of the following three procedures first the characterization of decision attributes given classes is extracted from databases and the classes are classified into several generalized groups with respect to the characterization then two kinds of sub rules classification rules for each generalized group and rules for each class within each group are induced finally those two parts are integrated into one rule for each decision attribute the proposed method was evaluated on a medical database the experimental results of which show that induced rules correctly represent experts decision processes

constructive and algebraic methods of the theory of rough sets this paper reviews and compares constructive and algebraic approaches in the study of rough sets in the constructive approach one starts from a binary relation and defines a pair of lower and upper approximation operators using the binary relation different classes of rough set algebras are obtained from different types of binary relations in the algebraic approach one defines a pair of dual approximation operators and states axioms that must be satisfied by the operators various classes of rough set algebras are characterized by different sets of axioms axioms of approximation operators guarantee the existence of certain types of binary relations producing the same operators

rough set theory applied to lattice theory in this paper we intend to study a connection between rough sets and lattice theory we introduce the concepts of upper and lower rough ideals filters in a lattice then we offer some of their properties with regard to prime ideals filters the set of all fixed points compact elements and homomorphisms

generalization performance of magnitude preserving semi supervised ranking with graph based regularization semi supervised ranking is a relatively new and important learning problem inspired by many applications we propose a novel graph based regularized algorithm which learns the ranking function in the semi supervised learning framework it can exploit geometry of the data while preserving the magnitude of the preferences the least squares ranking loss is adopted and the optimal solution of our model has an explicit form we establish error analysis of our proposed algorithm and demonstrate the relationship between predictive performance and intrinsic properties of the graph the experiments on three datasets for recommendation task and two quantitative structure activity relationship datasets show that our method is effective and comparable to some other state of the art algorithms for ranking

neighborhood operator systems and approximations this paper presents a framework for the study of generalizing the standard notion of equivalence relation in rough set approximation space with various categories of k step neighborhood systems based on a binary relation on a finite universe six families of binary relations are obtained and the corresponding six classes of k step neighborhood systems are derived extensions of pawlak s rough set approximation operators based on such neighborhood systems are proposed properties of neighborhood operator systems and rough set approximation operators are investigated and their connections are examined

graph based semi supervised learning and spectral kernel design in this paper we consider a framework for semi supervised learning using spectral decomposition based unsupervised kernel design we relate this approach to previously proposed semi supervised learning methods on graphs we examine various theoretical properties of such methods in particular we present learning bounds and derive optimal kernel representation by minimizing the bound based on the theoretical analysis we are able to demonstrate why spectral kernel design based methods can improve the predictive performance empirical examples are included to illustrate the main consequences of our analysis

a property of the functions and distributions annihilated by a locally integrable system of complex vector fields page introduction 387 1 basic notation and ingredients 389 2 approximation and constancy on the fibers of the classical solutions of the homogeneous equations 393 3 first consequences and interpretation of the fundamental theorem 398 4 distribution solutions of the homogeneous equations 402 5 regularity of the solutions of the homogeneous equations 409 6 example of a system that is analytic hypo elliptic but not co hypo elliptic 413 references 420

a rough margin based support vector machine by introducing the rough set theory into the support vector machine svm a rough margin based svm rmsvm is proposed to deal with the overfitting problem due to outliers similar to the classical svm the rmsvm searches for the separating hyper plane that maximizes the rough margin defined by the lower and upper margin in this way more data points are adaptively considered rather than the few extreme value points used in the classical svm in addition different support vectors may have different effects on the learning of the separating hyper plane depending on their positions in the rough margin points in the lower margin have more effects than those in the boundary of the rough margin from experimental results on six benchmark datasets the classification accuracy of this algorithm is improved without additional computational expense compared with the classical n svm

images of connected sets by semicontinuous multifunctions abstract the image of a connected set by an upper semicontinuous or a lower semicontinuous multifunction whose values are nonempty and connected is connected we prove this theorem in its most general setting and show its usefulness in various examples from optimization and nonlinear analysis

learning the kernel function via regularization we study the problem of finding an optimal kernel from a prescribed convex set of kernels k for learning a real valued function by regularization we establish for a wide variety of regularization functionals that this leads to a convex optimization problem and for square loss regularization we characterize the solution of this problem we show that although k may be an uncountable set the optimal kernel is always obtained as a convex combination of at most m 2 basic kernels where m is the number of data examples in particular our results apply to learning the optimal radial kernel or the optimal dot product kernel

rough set based 1 v 1 and 1 v r approaches to support vector machine multi classification support vector machines svms are essentially binary classifiers to improve their applicability several methods have been suggested for extending svms for multi classification including one versus one 1 v 1 one versus rest 1 v r and dagsvm in this paper we first describe how binary classification with svms can be interpreted using rough sets a rough set approach to svm classification removes the necessity of exact classification and is especially useful when dealing with noisy data next by utilizing the boundary region in rough sets we suggest two new approaches extensions of 1 v r and 1 v 1 to svm multi classification that allow for an error rate we explicitly demonstrate how our extended 1 v r may shorten the training time of the conventional 1 v r approach in addition we show that our 1 v 1 approach may have reduced storage requirements compared to the conventional 1 v 1 and dagsvm techniques our techniques also provide better semantic interpretations of the classification process the theoretical conclusions are supported by experimental findings involving a synthetic dataset

learning the kernel matrix with semidefinite programming kernel based learning algorithms work by embedding the data into a euclidean space and then searching for linear relations among the embedded data points the embedding is performed implicitly by specifying the inner products between each pair of points in the embedding space this information is contained in the so called kernel matrix a symmetric and positive semidefinite matrix that encodes the relative positions of all points specifying this matrix amounts to specifying the geometry of the embedding space and inducing a notion of similarity in the input space classical model selection problems in machine learning in this paper we show how the kernel matrix can be learned from data via semidefinite programming sdp techniques when applied to a kernel matrix associated with both training and test data this gives a powerful transductive algorithm using the labeled part of the data one can learn an embedding also for the unlabeled part the similarity between test points is inferred from training points and their labels importantly these learning problems are convex so we obtain a method for learning both the model class and the function without local minima furthermore this approach leads directly to a convex method for learning the 2 norm soft margin parameter in support vector machines solving an important open problem

graph based semi supervised learning by mixed label propagation with a soft constraint abstract in recent years various graph based algorithms have been proposed for semi supervised learning where labeled and unlabeled examples are regarded as vertices in a weighted graph and similarity between examples is encoded by the weight of edges however most of these methods cannot be used to deal with dissimilarity or negative similarity in this paper we propose a mixed label propagation model with a single soft constraint which can effectively handle positive similarity and negative similarity simultaneously as well as allow the labeled data to be relabeled specifically the soft mixed label propagation model is a fractional quadratic programming problem with a single quadratic constraint and we apply the global optimal algorithm 1 for solving it yielding an global optimal solution in a computational effort of o n 3 log 1 numerical comparisons with several existing methods for common test datasets and a class of collaborative filtering problems verify the effectiveness of the method

a nonparametric classification method based on k associated graphs graph is a powerful representation formalism that has been widely employed in machine learning and data mining in this paper we present a graph based classification method consisting of the construction of a special graph referred to as k associated graph which is capable of representing similarity relationships among data cases and proportion of classes overlapping the main properties of the k associated graphs as well as the classification algorithm are described experimental evaluation indicates that the proposed technique captures topological structure of the training data and leads to good results on classification task particularly for noisy data in comparison to other well known classification techniques the proposed approach shows the following interesting features 1 a new measure called purity is introduced not only to characterize the degree of overlap among classes in the input data set but also to construct the k associated optimal graph for classification 2 nonlinear classification with automatic local adaptation according to the input data contrasting to k nearest neighbor classifier which uses a fixed k the proposed algorithm is able to automatically consider different values of k in order to best fit the corresponding overlap of classes in different data subspaces revealing both the local and global structure of input data 3 the proposed classification algorithm is nonparametric implicating high efficiency and no need for model selection in practical applications