inference on the prediction of ensembles of infinite size in this paper we introduce a framework for making statistical inference on the asymptotic prediction of parallel classification ensembles the validity of the analysis is fairly general it only requires that the individual classifiers are generated in independent executions of some randomized learning algorithm and that the final ensemble prediction is made via majority voting given an unlabeled test instance the predictions of the classifiers in the ensemble are obtained sequentially as the individual predictions become known bayes theorem is used to update an estimate of the probability that the class predicted by the current ensemble coincides with the classification of the corresponding ensemble of infinite size using this estimate the voting process can be halted when the confidence on the asymptotic prediction is sufficiently high an empirical investigation in several benchmark classification problems shows that most of the test instances require querying only a small number of classifiers to converge to the infinite ensemble prediction with a high degree of confidence for these instances the difference between the generalization error of the finite ensemble and the infinite ensemble limit is very small often negligible

ensemble pruning via semi definite programming an ensemble is a group of learning models that jointly solve a problem however the ensembles generated by existing techniques are sometimes unnecessarily large which can lead to extra memory usage computational costs and occasional decreases in effectiveness the purpose of ensemble pruning is to search for a good subset of ensemble members that performs as well as or better than the original ensemble this subset selection problem is a combinatorial optimization problem and thus finding the exact optimal solution is computationally prohibitive various heuristic methods have been developed to obtain an approximate solution however most of the existing heuristics use simple greedy search as the optimization method which lacks either theoretical or empirical quality guarantees in this paper the ensemble subset selection problem is formulated as a quadratic integer programming problem by applying semi definite programming sdp as a solution technique we are able to get better approximate solutions computational experiments show that this sdp based pruning algorithm outperforms other heuristics in the literature its application in a classifier sharing study also demonstrates the effectiveness of the method

controllability of generalized linear time invariant systems this note deals with the problem of c controllability of generalized systems a new criterion for determining c controllability of generalized systems is established with controllability established it is shown that it is always possible to transfer the system from the initial position to any target by a control vector valued function whose elements are polynomials of bounded degree the polynomial coefficients are obtained as a solution to a set of linear equations

regression learning based on incomplete relationships between attributes abstract in recent years machine learning researchers have focused on methods to construct flexible and interpretable regression models however the method of obtaining complete knowledge from incomplete and fuzzy prior knowledge and the trade off between the generalization performance and the interpretability of the model are very important factors to consider in this paper we propose a new regression learning method complete relationships are obtained from the incomplete fuzzy relationships between attributes by using markov logic networks 29 the complete relationships are then applied to constrain the shape of the regression model in the optimization procedure to solve the trade off problem finally the benefits of our approach are illustrated on benchmark data sets and in real world experiments

robust output regulation of singular nonlinear systems via a nonlinear internal model the robust output regulation problem for singular nonlinear systems has been studied recently under a restrictive assumption that the solution of the regulator equations is polynomial this assumption essentially limits the nonlinear systems to contain only polynomial nonlinearities in this note we will further show that the polynomial assumption can be replaced by a much milder condition the new condition applies to a larger class of nonpolynomial nonlinear systems thus significantly improving the existing result

learning similarity with cosine similarity ensemble this paper proposes a cosine similarity ensemble cse method to learn similarity cse is a selective ensemble and combines multiple cosine similarity learners a learner redefines the pattern vectors and determines its threshold adaptively experimental results show the superiority of cse there is no doubt that similarity is a fundamental notion in the field of machine learning and pattern recognition how to represent and measure similarity appropriately is a pursuit of many researchers many tasks such as classification and clustering can be accomplished perfectly when a similarity metric is well defined cosine similarity is a widely used metric that is both simple and effective this paper proposes a cosine similarity ensemble cse method for learning similarity in cse diversity is guaranteed by using multiple cosine similarity learners each of which makes use of a different initial point to define the pattern vectors used in its similarity measures the cse method is not limited to measuring similarity using only pattern vectors that start at the origin in addition the thresholds of these separate cosine similarity learners are adaptively determined the idea of using a selective ensemble is also implemented in cse and the proposed cse method outperforms other compared methods on various data sets

ensemble pruning using spectral coefficients ensemble pruning aims to increase efficiency by reducing the number of base classifiers without sacrificing and preferably enhancing performance in this brief a novel pruning paradigm is proposed two class supervised learning problems are pruned using a combination of first and second order walsh coefficients a comparison is made with other ordered aggregation pruning methods using multilayer perceptron base classifiers the walsh pruning method is analyzed with the help of a model that shows the relationship between second order coefficients and added classification error with respect to bayes error

a theoretical study on six classifier fusion strategies we look at a single point in feature space two classes and l classifiers estimating the posterior probability for class spl omega sub 1 assuming that the estimates are independent and identically distributed normal or uniform we give formulas for the classification error for the following fusion methods average minimum maximum median majority vote and oracle

progressive subspace ensemble learning there are not many classifier ensemble approaches which investigate the data sample space and the feature space at the same time and this multi pronged approach will be helpful for constructing more powerful learning models for example the adaboost approach only investigates the data sample space while the random subspace technique only focuses on the feature space to address this limitation we propose the progressive subspace ensemble learning approach psel which takes into account the data sample space and the feature space at the same time specifically psel first adopts the random subspace technique to generate a set of subspaces then a progressive selection process based on new cost functions that incorporate current and long term information to select the classifiers sequentially will be introduced finally a weighted voting scheme is used to summarize the predicted labels and obtain the final result we also adopt a number of non parametric tests to compare psel and its competitors over multiple datasets the results of the experiments show that psel works well on most of the real datasets and outperforms a number of state of the art classifier ensemble approaches highlightsprogressive subspace ensemble learning it takes into account the data sample space and the feature space at the same time a progressive selection process based on new cost functions that incorporate current and long term information to select the classifiers sequentially will be introduced

wavelet decompositions of random forests smoothness analysis sparse approximation and applications in this paper we introduce in the setting of machine learning a generalization of wavelet analysis which is a popular approach to low dimensional structured signal analysis the wavelet decomposition of a random forest provides a sparse approximation of any regression or classification high dimensional function at various levels of detail with a concrete ordering of the random forest nodes from significant elements to nodes capturing only insignificant noise motivated by function space theory we use the wavelet decomposition to compute numerically a weak type smoothness index that captures the complexity of the underlying function as we show through extensive experimentation this sparse representation facilitates a variety of applications such as improved regression for difficult datasets a novel approach to feature importance resilience to noisy or irrelevant features compression of ensembles etc

a twin hyperspheres support vector machine with automatic variable weights for data classification abstract this paper proposes a novel twin hyperspheres support vector machine thsvm classifier for binary classification called the automatic variable weighted thsvm vthsvm classifier by solving a single optimization problem this classifier not only finds a pair of hyperspheres for classification but also automatically constructs a weight vector for each class in order to describe the dissimilarity of different classes this vthsvm is extended to the kernel case by the fact that a kernel can be written as a sum of one s evaluated on each variable separately the main advantage of this method is that it allows the use of adaptive distance which is suitable to find an as compact as possible hypersphere for each class experiments with synthetic and benchmark datasets indicate vthsvm obtains better performance than some other classifiers

exploration of classification confidence in ensemble learning ensemble learning has attracted considerable attention owing to its good generalization performance the main issues in constructing a powerful ensemble include training a set of diverse and accurate base classifiers and effectively combining them ensemble margin computed as the difference of the vote numbers received by the correct class and the another class received with the most votes is widely used to explain the success of ensemble learning this definition of the ensemble margin does not consider the classification confidence of base classifiers in this work we explore the influence of the classification confidence of the base classifiers in ensemble learning and obtain some interesting conclusions first we extend the definition of ensemble margin based on the classification confidence of the base classifiers then an optimization objective is designed to compute the weights of the base classifiers by minimizing the margin induced classification loss several strategies are tried to utilize the classification confidences and the weights it is observed that weighted voting based on classification confidence is better than simple voting if all the base classifiers are used in addition ensemble pruning can further improve the performance of a weighted voting ensemble we also compare the proposed fusion technique with some classical algorithms the experimental results also show the effectiveness of weighted voting with classification confidence

using unsupervised analysis to constrain generalization bounds for support vector classifiers a crucial issue in designing learning machines is to select the correct model parameters when the number of available samples is small theoretical sample based generalization bounds can prove effective provided that they are tight and track the validation error correctly the maximal discrepancy md approach is a very promising technique for model selection for support vector machines svm and estimates a classifier s generalization performance by multiple training cycles on random labeled data this paper presents a general method to compute the generalization bounds for svms which is based on referring the svm parameters to an unsupervised solution and shows that such an approach yields tight bounds and attains effective model selection when one estimates the generalization error one uses an unsupervised reference to constrain the complexity of the learning machine thereby possibly decreasing sharply the number of admissible hypothesis although the methodology has a general value the method described in the paper adopts vector quantization vq as a representation paradigm and introduces a biased regularization approach in bound computation and learning experimental results validate the proposed method on complex real world data sets

paper time domain input output representations of linear systems a non oriented matrix pencil model containing internal and external variables is shown to possess properties useful for modelling and design of linear dynamical systems three of these properties are trivial inversion invariance under a useful class of operations and simple decomposition using easily implemented row and column operations realizations of state space and polynomial operator models are given then a definition of system dimension and a minimal reduction algorithm a new time domain canonical form compatible with polynomial operator systems is described examples of the use of the model in algebraic design problems are given

an improved rounding method and semidefinite programming relaxation for graph partition given an undirected graph g v e with v n and an integer k between 0 and n the maximization graph partition max gp problem is to determine a subset s v of k nodes such that an objective function w s is maximized the max gp problem can be formulated as a binary quadratic program and it is np hard semidefinite programming sdp relaxations of such quadratic programs have been used to design approximation algorithms with guaranteed performance ratios for various max gp problems based on several earlier results we present an improved rounding method using an sdp relaxation and establish improved approximation ratios for several max gp problems including dense subgraph max cut max not cut and max vertex cover

regularity and stability analysis of discrete time markov jump linear singular systems abstract in this paper the regularity and stability analysis of discrete time markov jump linear singular systems mjlss is performed when dealing with singular systems a primary concern is related to the existence and uniqueness of a solution to the system this problem that is called regularity problem has a known solution when the linear singular system is not subject to jumps lss it turns out that when the pair of matrices that describes the dynamics of the lss satisfies a certain condition then it is regular by extending this condition to mjlss a unique solution for this class of systems is derived indeed through the idea of mode to mode regularity which is stronger than the mode by mode notion that can be found in the literature the existence of a unique solution of an mjlss is shown furthermore for systems that are mode to mode regular it is shown how to obtain recursive second moment models whose complexity depends on how anticipative the systems is the derived results on regularity and second moment modeling enable us to study stability of an mjlss by following the literature on markov jump linear systems mjls four different concepts of stability are introduced and their relationship is established the results presented in this paper generalize well known results concerning the stability given in the mjls theory for instance it is shown that the stability of the system is equivalent to the spectral radius of an augmented matrix being less than one as happens in the theory of mjls

linear programming support vector machines abstract based on the analysis of the conclusions in the statistical learning theory especially the vc dimension of linear functions linear programming support vector machines or svms are presented including linear programming linear and nonlinear svms in linear programming svms in order to improve the speed of the training time the bound of the vc dimension is loosened properly simulation results for both artificial and real data show that the generalization performance of our method is a good approximation of svms and the computation complex is largely reduced by our method

eros ensemble rough subspaces ensemble learning is attracting much attention from pattern recognition and machine learning domains for good generalization both theoretical and experimental researches show that combining a set of accurate and diverse classifiers will lead to a powerful classification system an algorithm called fs pp eros for selective ensemble of rough subspaces is proposed in this paper rough set based attribute reduction is introduced to generate a set of reducts and then each reduct is used to train a base classifier we introduce an accuracy guided forward search and post pruning strategy to select part of the base classifiers for constructing an efficient and effective ensemble system the experiments show that classification accuracies of ensemble systems with accuracy guided forward search strategy will increase at first arrive at a maximal value then decrease in sequentially adding the base classifiers we delete the base classifiers added after the maximal accuracy the experimental results show that the proposed ensemble systems outperform bagging and random subspace methods in terms of accuracy and size of ensemble systems fs pp eros can keep or improve the classification accuracy with very few base classifiers which leads to a powerful and compact classification system

boosting the margin a new explanation for the effectiveness of voting methods one of the surprising recurring phenomena observed in experiments with boosting is that the test error of the generated classifier usually does not increase as its size becomes very large and often is observed to decrease even after the training error reaches zero in this paper we show that this phenomenon is related to the distribution of margins of the training examples with respect to the generated voting classification rule where the margin of an example is simply the difference between the number of correct votes and the maximum number of votes received by any incorrect label we show that techniques used in the analysis of vapnik s support vector classifiers and of neural networks with small weights can be applied to voting methods to relate the margin distribution to the test error we also show theoretically and experimentally that boosting is especially effective at increasing the margins of the training examples finally we compare our explanation to those based on the bias variance decomposition

a nonlinear programming algorithm for solving semidefinite programs via low rank factorization in this paper we present a nonlinear programming algorithm for solving semidefinite programs sdps in standard form the algorithm s distinguishing feature is a change of variables that replaces the symmetric positive semidefinite variable x of the sdp with a rectangular variable r according to the factorization x rr t the rank of the factorization i e the number of columns of r is chosen minimally so as to enhance computational speed while maintaining equivalence with the sdp fundamental results concerning the convergence of the algorithm are derived and encouraging computational results on some large scale test problems are also presented

on the analysis of discrete linear time invariant singular systems the discrete singular equation over an interval can represent a two point boundary value problem or it can be considered as a dynamical relation developing forward in time a theory is provided by giving analytic solutions and discussing system properties in both cases that encompasses both interpretations the singular system fundamental matrix is used to provide analytic solutions to a time invariant discrete singular equation defined over an interval the two distinct cases in which the singular relation is interpreted as a two point boundary value problem or as a forward dynamical system on the interval are both considered also considered is the case in which the singular relation is considered as a backward dynamical system and the relationship between the index of nilpotence and the laurent series coefficients resulting from the solutions of certain state space equations is shown reachability and observability are discussed and the point is made that these properties are different depending on how the singular relation is interpreted

lyapunov and riccati equations of discrete time descriptor systems in this paper we further develop the generalized lyapunov equations for discrete time descriptor systems given by bender we associate a stable discrete time descriptor system with a lyapunov equation which has a unique solution furthermore under the assumptions of reachability and observability the solutions are guaranteed to be positive definite all results are valid for causal and noncausal descriptor systems this provides a unification of lyapunov equations and theories established for both normal and descriptor systems based on the developed lyapunov equation a riccati equation is also obtained for solving the state feedback stabilization problem

solvability controllability and observability of continuous descriptor systems in this paper we investigate the properties of the continuous descriptor system e dot x t ax t bu t 0 leqt leqb where e a and b are complex and possibly singular matrices and u t is a complex function differentiable sufficiently many times the traditional approach to such systems is to separate the state equations from the algebraic equations however such algorithms usually destroy the natural physically based sparsity and structure of the original system therefore we consider descriptor systems in their original form such systems possess numerous properties not shared by the well known state variable systems first we relate classical theories of matrix pencils to the solvability of descriptor systems then we extend the concepts of reachability controllability and observability of state variable systems to descriptor systems and describe the set of reachable states for descriptor systems

locally adaptive metric nearest neighbor classification nearest neighbor classification assumes locally constant class conditional probabilities this assumption becomes invalid in high dimensions with finite samples due to the curse of dimensionality severe bias can be introduced under these conditions when using the nearest neighbor rule we propose a locally adaptive nearest neighbor classification method to try to minimize bias we use a chi squared distance analysis to compute a flexible metric for producing neighborhoods that are highly adaptive to query locations neighborhoods are elongated along less relevant feature dimensions and constricted along most influential ones as a result the class conditional probabilities are smoother in the modified neighborhoods whereby better classification performance can be achieved the efficacy of our method is validated and compared against other techniques using both simulated and real world data

optimizing the data dependent kernel under a unified kernel optimization framework the kernel functions play a central role in kernel methods accordingly over the years the optimization of kernel functions has been a promising research area ideally fisher discriminant criteria can be used as an objective function to optimize the kernel function to augment the margin between different classes unfortunately fisher criteria are optimal only in the case that all the classes are generated from underlying multivariate normal distributions of common covariance matrix but different means and each class is expressed by a single cluster due to the assumptions fisher criteria obviously are not a suitable choice as a kernel optimization rule in some applications such as the multimodally distributed data in order to solve this problem recently many improved discriminant criteria dc have been also developed therefore to apply these discriminant criteria to kernel optimization in this paper based on a data dependent kernel function we propose a unified kernel optimization framework which can use any discriminant criteria formulated in a pairwise manner as the objective functions under the kernel optimization framework to employ different discriminant criteria one has to only change the corresponding affinity matrices without having to resort to any complex derivations in feature space experimental results based on some benchmark data demonstrate the efficiency of our method

selective ensemble of svdds with renyi entropy based diversity measure abstract in this paper a novel selective ensemble strategy for support vector data description svdd using the renyi entropy based diversity measure is proposed to deal with the problem of one class classification in order to obtain compact classification boundary the radius of ensemble is defined as the inner product of the vector of combination weights and the vector of the radii of svdds to make the center of ensemble achieve the optimal position the renyi entropy of the kernelized distances between the images of samples and the center of ensemble in the high dimensional feature space is defined as the diversity measure moreover to fulfill the selective ensemble an l 1 norm based regularization term is introduced into the objective function of the proposed ensemble the optimal combination weights can be iteratively obtained by the half quadratic optimization technique experimental results on two synthetic data sets and twenty benchmark data sets demonstrate that the proposed selective ensemble method is superior to the single svdd and the other four related ensemble approaches

density induced margin support vector machines this paper proposes a new classifier called density induced margin support vector machines dmsvms dmsvms belong to a family of svm like classifiers thus dmsvms inherit good properties from support vector machines svms e g unique and global solution and sparse representation for the decision function for a given data set dmsvms require to extract relative density degrees for all training data points these density degrees can be taken as relative margins of corresponding training data points moreover we propose a method for estimating relative density degrees by using the k nearest neighbor method we also show the upper bound on the leave out one error of dmsvms for a binary classification problem and prove it promising results are obtained on toy as well as real world data sets

fundamental reachability and observability matrices for discrete descriptor systems this paper uses the fundamental matrix of a regular discrete descriptor system to derive expressions for descriptor reachability and observability matrices reachable and unobservable subspaces are defined it is shown that the natural space for analyzing descriptor system properties is r 2n where n is the dimension of the system not rnas is the case for state space systems solutions are provided for the descriptor open loop control and estimation problems

combining neighborhood separable subspaces for classification via sparsity regularized optimization the neighborhood rough set theory has been successfully applied to various classification tasks the key concept of this theory is to find a sufficient and necessary neighborhood separable subspace for building a compact model given a classification learning task there usually exist numerous neighborhood separable subspaces that maintain the discriminative ability of the original space with respect to a given granularity these subspaces contain complementary information for classification however it is a challenging task to compute these subspaces efficiently in this paper we develop a fast neighborhood attribute reduction algorithm based on sample pair selection to find all reducts nevertheless it cannot deal with large scale data then we propose a randomized attribute reduction algorithm based on neighborhood dependency the randomized algorithm can find a part of all reducts and is very efficient a classification framework of joint subspace representation is proposed to fully exploit the complementary information in different subspaces in addition a weight matrix is learned to combine the representation residuals in the different subspaces via group sparsity regularization the performances of the proposed attribute reduction algorithms are compared and the influence of granularity on attribute reduction is discussed finally the proposed technique is compared with other ensemble learning algorithms experimental results show that the proposed framework is superior to state of the art classifiers

how large should ensembles of classifiers be we propose to determine the size of a parallel ensemble by estimating the minimum number of classifiers that are required to obtain stable aggregate predictions assuming that majority voting is used a statistical description of the convergence of the ensemble prediction to its asymptotic infinite size limit is given the analysis of the voting process shows that for most test instances the ensemble prediction stabilizes after only a few classifiers are polled by contrast a small but non negligible fraction of these instances require large numbers of classifier queries to reach stable predictions specifically the fraction of instances whose stable predictions require more than t classifiers for email protected 1 has a universal form and is proportional to t 1 2 the ensemble size is determined as the minimum number of classifiers that are needed to estimate the infinite ensemble prediction at an average confidence level a close to one this approach differs from previous proposals which are based on determining the size for which the prediction error not the predictions themselves stabilizes in particular it does not require estimates of the generalization performance of the ensemble which can be unreliable it has general validity because it is based solely on the statistical description of the convergence of majority voting to its asymptotic limit extensive experiments using representative parallel ensembles bagging and random forest illustrate the application of the proposed framework in a wide range of classification problems these experiments show that the optimal ensemble size is very sensitive to the particular classification problem considered

the weighted majority algorithm we study the construction of prediction algorithms in a situation in which a learner faces a sequence of trials with a prediction to be made in each and the goal of the learner is to make few mistakes we are interested in the case where the learner has reason to believe that one of some pool of known algorithms will perform well but the learner does not know which one a simple and effective method based on weighted voting is introduced for constructing a compound algorithm in such a circumstance we call this method the weighted majority algorithm we show that this algorithm is robust in the presence of errors in the data we discuss various versions of the weighted majority algorithm and prove mistake bounds for them that are closely related to the mistake bounds of the best algorithms of the pool for example given a sequence of trials if there is an algorithm in the pool a that makes at most m mistakes then the weighted majority algorithm will make at most c log a m mistakes on that sequence where c is fixed constant

induction of fuzzy decision trees abstract most decision tree induction methods used for extracting knowledge in classification problems do not deal with cognitive uncertainties such as vagueness and ambiguity associated with human thinking and perception in this paper cognitive uncertainties involved in classification problems are explicitly represented measured and incorporated into the knowledge induction process a fuzzy decision tree induction method which is based on the reduction of classification ambiguity with fuzzy evidence is developed fuzzy decision trees represent classification knowledge more naturally to the way of human thinking and are more robust in tolerating imprecise conflict and missing information

on the stopping criteria for k nearest neighbor in positive unlabeled time series classification problems positive unlabeled time series classification has become an important area during the last decade as often vast amounts of unlabeled time series data are available but obtaining the corresponding labels is difficult in this situation positive unlabeled learning is a suitable option to mitigate the lack of labeled examples in particular self training is a widely used technique due to its simplicity and adaptability within this technique the stopping criterion i e the decision of when to stop labeling is a critical part especially in the positive unlabeled context we propose a self training method that follows the positive unlabeled approach for time series classification and a family of parameter free stopping criteria for this method our proposal uses a graphical analysis applied to the minimum distances obtained by the k nearest neighbor as the base learner to estimate the class boundary the proposed method is evaluated in an experimental study involving various time series classification datasets the results show that our method outperforms the transductive results obtained by previous models

the generalization ability of online algorithms for dependent data we study the generalization performance of online learning algorithms trained on samples coming from a dependent source of data we show that the generalization error of any stable online algorithm concentrates around its regret an easily computable statistic of the online performance of the algorithm when the underlying ergodic process is or mixing we show high probability error bounds assuming the loss function is convex and we also establish sharp convergence rates and deviation bounds for strongly convex losses and several linear prediction problems such as linear and logistic regression least squares svm and boosting on dependent data in addition our results have straightforward applications to stochastic optimization with dependent data and our analysis requires only martingale convergence arguments we need not rely on more powerful statistical tools such as empirical process theory

dimensionality reduction via sparse support vector machines we describe a methodology for performing variable ranking and selection using support vector machines svms the method constructs a series of sparse linear svms to generate linear models that can generalize well and uses a subset of nonzero weighted variables found by the linear models to produce a final nonlinear model the method exploits the fact that a linear svm no kernels with l1 norm regularization inherently performs variable selection as a side effect of minimizing capacity of the svm model the distribution of the linear model weights provides a mechanism for ranking and interpreting the effects of variables starplots are used to visualize the magnitude and variance of the weights for each variable we illustrate the effectiveness of the methodology on synthetic data benchmark problems and challenging regression problems in drug design this method can dramatically reduce the number of variables and outperforms svms trained using all attributes and using the attributes selected according to correlation coefficients the visualization of the resulting models is useful for understanding the role of underlying variables

periodic descriptor systems solvability and conditionability the authors consider discrete time linear periodic descriptor systems and study the concepts of solvability and conditionability introduced by luenberger they prove that solvability is equivalent to conditionability just as in the time invariant case we give a characterization of solvability conditionability in terms of a cyclic matrix pencil and furthermore propose a simple test via the periodic schur decomposition to check for either property this could lead to further systematic study of these systems

an analysis on the relationship between uncertainty and misclassification rate of classifiers abstract this paper provides new insight into the analysis on the relationship between uncertainty and misclassification of a classifier we formulate the relationship explicitly by taking entropy as a measurement of uncertainty and by analyzing the misclassification rate based on the membership degree difference focusing on binary classification problems this study theoretically and experimentally validates that the misclassification rate will definitely be upgrading with the increase of uncertainty if two conditions are satisfied 1 the distributions of two classes based on membership degree difference are unimodal and 2 these two distributions attain peaks when the membership degree difference is less and larger than zero respectively this work aims to provide some practical guidelines for improving classifier performance through clearly expressing and understanding the relationship between uncertainty and misclassification of a classifier

domains of competence of fuzzy rule based classification systems with data complexity measures a case of study using a fuzzy hybrid genetic based machine learning method the analysis of data complexity is a proper framework to characterize the tackled classification problem and to identify domains of competence of classifiers as a practical outcome of this framework the proposed data complexity measures may facilitate the choice of a classifier for a given problem the aim of this paper is to study the behaviour of a fuzzy rule based classification system and its relationship to data complexity we use as a case of study the fuzzy hybrid genetic based machine learning method presented in h ishibuchi t yamamoto t nakashima hybridization of fuzzy gbml approaches for pattern classification problems ieee transactions on systems man and cybernetics part b cybernetics 35 2 2005 359 365 we examine several metrics of data complexity over a wide range of data sets built from real data and try to extract behaviour patterns from the results we obtain rules which describe both good or bad behaviours of the fuzzy rule based classification system these rules use values of data complexity metrics in their antecedents so we try to predict the behaviour of the method from the data set complexity metrics prior to its application therefore we can establish the domains of competence of this fuzzy rule based classification system

the random subspace method for constructing decision forests much of previous attention on decision trees focuses on the splitting criteria and optimization of tree sizes the dilemma between overfitting and achieving maximum accuracy is seldom resolved a method to construct a decision tree based classifier is proposed that maintains highest accuracy on training data and improves on generalization accuracy as it grows in complexity the classifier consists of multiple trees constructed systematically by pseudorandomly selecting subsets of components of the feature vector that is trees constructed in randomly chosen subspaces the subspace method is compared to single tree classifiers and other forest construction methods by experiments on publicly available datasets where the method s superiority is demonstrated we also discuss independence between trees in a forest and relate that to the combined classification accuracy

existence and uniqueness of solutions of linear variable coefficient discrete time descriptor systems abstract we consider linear discrete time descriptor systems i e systems of linear equations of the form e k x k 1 a k x k f k for k z where all e k and a k are matrices f k are vectors and x k are the vectors of the solution we are looking for we study the existence and uniqueness of solutions a strangeness index is defined for such systems compared to the continuous time case see p kunkel v mehrmann differential algebraic equations analysis and numerical solution european mathematical society zurich 2006 in the discrete time case we have to account for the fact that it makes a difference if one has an initial condition and one wants a solution in the future or if one has an initial condition and one wants a solution into the past and the future at the same time

sparse ensembles using weighted combination methods based on linear programming an ensemble of multiple classifiers is widely considered to be an effective technique for improving accuracy and stability of a single classifier this paper proposes a framework of sparse ensembles and deals with new linear weighted combination methods for sparse ensembles sparse ensemble is to sparsely combine the outputs of multiple classifiers by using a sparse weight vector when the continuous outputs of multiple classifiers are provided in our methods the problem of solving sparse weight vector can be formulated as linear programming problems in which the hinge loss or and the 1 norm regularization are exploited both the hinge loss and the 1 norm regularization are techniques inducing sparsity used in machine learning we only ensemble classifiers with nonzero weight coefficients in these lp based methods the ensemble training error is minimized while the weight vector of ensemble learning is controlled which can be thought as implementing the structure risk minimization rule and naturally explains good performance of these methods the promising experimental results over uci data sets and the radar high resolution range profile data are presented

regularization theory and neural networks architectures we had previously shown that regularization principles lead to approximation schemes that are equivalent to networks with one layer of hidden units called regularization networks in particular s

a comfort measure for diagnostic problem solving abstract in order to apply bayes theorem for diagnostic problem solving when multiple disorders can occur simultaneously several previous proposals have suggested using the ratio of posterior probabilities to rank diagnostic hypotheses approaches using such relative likelihoods lose the measure of absolute strengths of hypotheses and thus are incapable of evaluating the quality of a problem solution in this paper we propose to impose a quantity called a comfort measure on the solution a solution of a diagnostic problem is a minimal size set of hypotheses such that the sum of their posterior probabilities exceeds a given comfort measure based on a probabilistic causal model developed previously a problem solving strategy is presented which does not require the manifestation independence assumption required with direct bayesian classification and which is applicable to multimembership classification problems this strategy selectively generates diagnostic hypotheses and calculates both their relative likelihood and the lower and upper bounds of their posterior probabilities these bounds are successively refined as more hypotheses are generated using these bounds not the real posterior probabilities the problem solving strategy identifies a solution satisfying the given comfort measure usually after only a small portion of all possible hypotheses have been generated

random forests with ensemble of feature spaces random forests receive much attention from researchers because of their excellent performance as breiman suggested the performance of random forests depends on the strength of the weak learners in the forests and the diversity among them however in the literature many researchers only considered pre processing of the data or post processing of the random forests models in this paper we propose a new method to increase the diversity of each tree in the forests and thereby improve the overall accuracy during the training process of each individual tree in the forest different rotation spaces are concatenated into a higher space at the root node then the best split is exhaustively searched within this higher space the location where the best split lies decides which rotation method to be used for all subsequent nodes the performance of the proposed method here is evaluated on 42 benchmark data sets from various research fields and compared with the standard random forests the results show that the proposed method improves the performance of the random forests in most cases

a novel combining classifier method based on variational inference in this paper we propose a combining classifier method based on the bayesian inference framework specifically the outputs of base classifiers called level1 data or meta data are utilized in a combiner to produce the final classification in our ensemble system each class in the training set induces a distribution on the level1 data which is modeled by a multivariate gaussian distribution traditionally the parameters of the gaussian are estimated using a maximum likelihood approach however maximum likelihood estimation cannot be applied since the covariance matrix of level1 data of each class is not full rank instead we propose to estimate the multivariate gaussian distribution of level1 data of each class by using the variational inference method experiments conducted on eighteen uci machine learning repository datasets and a selected 10 class clef2009 medical imaging database demonstrated the advantage of our method compared with several well known ensemble methods we proposed a variational inference method for multivariate gaussian distribution estimation we proposed a novel combining classifier method by employing variational inference on base classifiers outputs the proposed method is highly competitive to many state of the arts ensemble classifier methods

analysis of descriptor systems using numerical algorithms in this paper we analyze numerical methods for the solution of the large scale dynamical system e dot y t ay t g t y t 0 y 0 where e and a are matrices possibly singular systems of this type have been referred to as implicit systems and more recently as descriptor systems since they arise from formulating system equations in physical variables special cases of such systems are algebraic differential systems we discuss the numerical advantages of this formulation and identify a class of numerical integration algorithms which have accuracy and stability properties appropriate to descriptor systems and which preserve structure detect nonsolvable systems resolve initial value consistency problems and are applicable to stiff descriptor systems we also present an algorithm for the control of the local truncation error on only the state variables

an equivalence between sparse approximation and support vector machines in the first part of this paper we show a similarity between the principle of structural risk minimization principle srm vapnik 1982 and the idea of sparse approximation as defined in chen donoho and saunders 1995 and olshausen and field 1996 then we focus on two specific approximate implementations of srm and sparse approximation which have been used to solve the problem of function approximation for srm we consider the support vector machine technique proposed by v vapnik and his team at at t bell labs and for sparse approximation we consider a modification of the basis pursuit de noising algorithm proposed by chen donoho and saunders 1995 we show that under certain conditions these two techniques are equivalent they give the same solution and they require the solution of the same quadratic programming problem

complexity measures of supervised classification problems we studied a number of measures that characterize the difficulty of a classification problem focusing on the geometrical complexity of the class boundary we compared a set of real world problems to random labelings of points and found that real problems contain structures in this measurement space that are significantly different from the random sets distributions of problems in this space show that there exist at least two independent factors affecting a problem s difficulty we suggest using this space to describe a classifier s domain of competence this can guide static and dynamic selection of classifiers for specific problems as well as subproblems formed by confinement projection and transformations of the feature vectors

fuzzy support vector machines a support vector machine svm learns the decision surface from two distinct classes of the input points in many applications each input point may not be fully assigned to one of these two classes in this paper we apply a fuzzy membership to each input point and reformulate the svms such that different input points can make different contributions to the learning of decision surface we call the proposed method fuzzy svms fsvms

shared domains of competence of approximate learning models using measures of separability of classes in this work we jointly analyze the performance of three classic artificial neural network models and one support vector machine with respect to a series of data complexity measures known as measures of separability of classes in particular we consider a radial basis function network a multi layer perceptron a learning vector quantization while the sequential minimal optimization method is used to model the support vector machine we consider five measures of separability of classes over a wide range of data sets built from real data which have proved to be very discriminative when analyzing the performance of classifiers we find that two of them allow us to extract common behavior patterns for the four learning methods due to their related nature we obtain rules using these two metrics that describe both good or bad performance of the artificial neural networks and the support vector machine with the obtained rules we characterize the performance of the methods from the data set complexity metrics and therefore their common domains of competence are established using these domains of competence the shared good and bad capabilities of these four models can be used to know if the approximative models will perform well or poorly or if a more complex configuration of the model is needed for a given problem in advance

the computational complexity of probabilistic inference using bayesian belief networks research note abstract bayesian belief networks provide a natural efficient method for representing probabilistic dependencies among a set of variables for these reasons numerous researchers are exploring the use of belief networks as a knowledge representation in artificial intelligence algorithms have been developed previously for efficient probabilistic inference using special classes of belief networks more general classes of belief networks however have eluded efforts to develop efficient inference algorithms we show that probabilistic inference using belief networks is np hard therefore it seems unlikely that an exact algorithm can be developed to perform probabilistic inference efficiently over all classes of belief networks this result suggests that research should be directed away from the search for a general efficient probabilistic inference algorithm and toward the design of efficient special case average case and approximation algorithms

improving generalization of fuzzy if then rules by maximizing fuzzy entropy when fuzzy if then rules initially extracted from data have not a satisfying performance we consider that the rules require refinement distinct from most existing rule refinement approaches that are based on the further reduction of training error this paper proposes a new rule refinement scheme that is based on the maximization of fuzzy entropy on the training set the new scheme which is realized by solving a quadratic programming problem is expected to have the advantages of improving the generalization capability of initial fuzzy if then rules and simultaneously overcoming the overfitting of refinement experimental results on a number of selected databases demonstrate the expected improvement of generalization capability and the prevention of overfitting by a comparison of both training and testing accuracy before and after the refinement

a formal model of diagnostic inference i problem formulation and decomposition abstract this paper which is part i of a two part series introduces a new model of diagnostic problem solving based on a generalization of the set covering problem the model formalizes the concepts of 1 1 whether or not a set of one or more disorders is sufficient to explain a set of occurring manifestations 2 2 what a solution is for a diagnostic problem and 3 3 how to generate all of the alternative explanations in a problem s solution in addition conditions for decomposing a diagnostic problem into independent subproblems are stated and proven this model is of interest because it captures several intuitively plausible features of human diagnostic inference it directly addresses the issue of multiple simultaneous causative disorders it can serve as a theoretical basis for expert systems for diagnostic problem solving and it provides a conceptual framework within which to view some recent ai work on diagnostic problem solving in general in part ii the concepts developed in this paper will be used to present algorithms for diagnostic problem solving

sequential combination of weighted and nonparametric bagging for classification we propose a simple sequential procedure for bagged classification which modifies nonparametric bagging by randomizing class labels of resampled data points the random labelling feature of the procedure also enables us to undertake unsupervised classification with the benefit of supervised learning theoretical properties are given for the nearest neighbour classifier in the case of supervised learning and a hard thresholding indicator in the case of unsupervised learning showing that sequential bagging accelerates convergence of the bagged predictor to the bayes rule simulation results are provided in support of the proposed method

universal approximation using incremental constructive feedforward networks with random hidden nodes according to conventional neural network theories single hidden layer feedforward networks slfns with additive or radial basis function rbf hidden nodes are universal approximators when all the parameters of the networks are allowed adjustable however as observed in most neural network implementations tuning all the parameters of the networks may cause learning complicated and inefficient and it may be difficult to train networks with nondifferential activation functions such as threshold networks unlike conventional neural network theories this paper proves in an incremental constructive method that in order to let slfns work as universal approximators one may simply randomly choose hidden nodes and then only need to adjust the output weights linking the hidden layer and the output layer in such slfns implementations the activation functions for additive nodes can be any bounded nonconstant piecewise continuous functions g r r and the activation functions for rbf nodes can be any integrable piecewise continuous functions g r r and rg x dx 0 the proposed incremental method is efficient not only for sflns with continuous including nondifferentiable activation functions but also for slfns with piecewise continuous such as threshold activation functions compared to other popular methods such a new network is fully automatic and users need not intervene the learning process by manually tuning control parameters

observability of linear discrete time systems of algebraic and difference equations abstractthe notion of observability for higher order discrete time systems of algebraic and difference equations is studied such systems are also known as polynomial matrix descriptions attention is first given to a special form of descriptor systems with a state lead in the output this system is transformed into its causal and noncausal subsystems and observability criteria are given in terms of the subsystem s matrices and the fundamental matrix sequence of the matrix pencil e a afterwards the higher order system is studied by transforming it into a first order descriptor system of the above form an observability criterion is provided for the higher order system in terms of the laurent expansion at infinity of the system s polynomial matrix in addition observability is connected with the coprimeness of the polynomial matrices of the higher order system and the coprimeness of the matrix pencils of the descriptor system

where do we stand on measures of uncertainty ambiguity fuzziness and the like abstract it is argued in this paper that the theory of fuzzy sets involves at least four fundamentally different types of uncertainty each of these types requires a measure by which the degree of uncertainty of that type can be determined two main categories of uncertainty are connected with the terms vagueness or fuzziness and ambiguity in general vagueness is associated with the difficulty of making sharp or precise distinctions in the world ambiguity on the other hand is associated with one to many relations i e situations with two or more alternatives that are left unspecified while the concept of a fuzzy set represents a basic mathematical framework for dealing with vagueness the concept of a fuzzy measure is a general framework for dealing with ambiguity several classes of measures of vagueness usually referred to as measures of fuzziness have been proposed in the literature each class is based on some underlying conception of the degree of fuzziness a general set of requirements for measures of fuzziness is formulated followed by an overview of the measures proposed in the literature measures of ambiguity are discussed within the framework of plausibility and belief measures although it does not cover all fuzzy measures this framework is sufficiently broad for most practical purposes and represents a generalization of both probability theory and possibility theory it is argued that three complementary measures of ambiguity should be employed one of them is obtained by generalizing the hartley measure of uncertainty it measures the degree of nonspecificity in individual situations described by the various belief and plausibility measures the other two are obtained by generalizing the well known shannon measure of uncertainty they measure the degree of dissonance and the degree of confusion in evidence respectively basic mathematical properties of these measures are overviewed it is also argued that each of the four types of uncertainty measures which are fundamentally different from each other can be used for measuring structural syntactic information in the same sense as the hartley and shannon measures have been used in this respect as such these measures are potentially powerful tools for dealing with systems problems such as systems modelling analysis simplification or design

computation of structural invariants of generalized state space systems in this paper we develop an algorithm for computing the zeros of a generalized state space model described by the matrix 5 tuple e a b c d where e may be a singular matrix but det a he 0 the characterization of these zeros is based on the system matrix of the corresponding 5 tuple both the characterization and the computational algorithm are extensions of equivalent results for state space models described by the 4 tuples a b c d we also extend these results to the computation of infinite zeros and left and right minimal indices of the system matrix several non trivial numerical examples are included to illustrate the proposed results

descriptor systems decomposition into forward and backward subsystems it is shown that a decomposition of the descriptor space with certain properties induces a decomposition of the descriptor system into forward and backward subsystems different decompositions correspond to different apportionments of the required boundary conditions between initial and final times applications include several different but equivalent methods for solving the descriptor system and a clarification of the relations between some previously known methods of solution

learning the kernel matrix with semidefinite programming kernel based learning algorithms work by embedding the data into a euclidean space and then searching for linear relations among the embedded data points the embedding is performed implicitly by specifying the inner products between each pair of points in the embedding space this information is contained in the so called kernel matrix a symmetric and positive semidefinite matrix that encodes the relative positions of all points specifying this matrix amounts to specifying the geometry of the embedding space and inducing a notion of similarity in the input space classical model selection problems in machine learning in this paper we show how the kernel matrix can be learned from data via semidefinite programming sdp techniques when applied to a kernel matrix associated with both training and test data this gives a powerful transductive algorithm using the labeled part of the data one can learn an embedding also for the unlabeled part the similarity between test points is inferred from training points and their labels importantly these learning problems are convex so we obtain a method for learning both the model class and the function without local minima furthermore this approach leads directly to a convex method for learning the 2 norm soft margin parameter in support vector machines solving an important open problem

incremental construction of classifier and discriminant ensembles we discuss approaches to incrementally construct an ensemble the first constructs an ensemble of classifiers choosing a subset from a larger set and the second constructs an ensemble of discriminants where a classifier is used for some classes only we investigate criteria including accuracy significant improvement diversity correlation and the role of search direction for discriminant ensembles we test subset selection and trees fusion is by voting or by a linear model using 14 classifiers on 38 data sets incremental search finds small accurate ensembles in polynomial time the discriminant ensemble uses a subset of discriminants and is simpler interpretable and accurate we see that an incremental ensemble has higher accuracy than bagging and random subspace method and it has a comparable accuracy to adaboost but fewer classifiers

consensus and complementarity based maximum entropy discrimination for multi view classification maximum entropy discrimination med is a general framework for discriminative estimation based on the maximum entropy and large margin principles but it just uses only one view of the data not all the views of the data although multi view maximum entropy discrimination considered the multi view information of the data it just respects the consensus principle there are two common principles named consensus and complementarity in multi view learning we aim to take the full advantage of the multi view information of the data for classification and respect the two common principles consensus and complementarity simultaneously in this paper we propose a new method called consensus and complementarity based med med 2c for multi view classification which well utilizes the two principles consensus and complementarity for multi view learning mvl we first transform data from two views into a common subspace and make the transformed data in the new subspace identical to respect the consensus principle then we augment the transformed data with their original features to take into account the complementarity principle built on the augmented features and meanwhile by relaxing the consensus principle the objective function of med 2c is naturally formed whose inner optimization recovers the traditional med framework we provide an instantiation of the med 2c method and derive the corresponding solution experimental results on synthetic data and real world data show the effectiveness of the proposed med 2c it not only performs better than three single view classification methods but also generally outperforms three multi view classification methods canonical correlation analysis cca ensemble med emed and the multi view variant svm 2k of the classical support vector machine svm algorithm in addition med 2c performs better than or as well as state of the art mvmed on all the data sets

decision combination in multiple classifier systems a multiple classifier system is a powerful solution to difficult pattern recognition problems involving large class sets and noisy input because it allows simultaneous use of arbitrary feature descriptors and classification procedures decisions by the classifiers can be represented as rankings of classifiers and different instances of a problem the rankings can be combined by methods that either reduce or rerank a given set of classes an intersection method and union method are proposed for class set reduction three methods based on the highest rank the borda count and logistic regression are proposed for class set reranking these methods have been tested in applications of degraded machine printed characters and works from large lexicons resulting in substantial improvement in overall correctness

on cutting planes for cardinality constrained linear programs we derive cutting planes for cardinality constrained linear programs these inequalities can be used to separate any basic feasible solution of an lp relaxation of the problem assuming that this solution violates the cardinality requirement to derive them we first relax the given simplex tableau into a disjunctive set expressed in the space of nonbasic variables we establish that coefficients of valid inequalities for the closed convex hull of this set obey ratios that can be computed directly from the simplex tableau we show that a transportation problem can be used to separate these inequalities we then give a constructive procedure to generate violated facet defining inequalities for the closed convex hull of the disjunctive set using a variant of prim s algorithm

boundary value descriptor systems well posedness reachability and observability in this paper we introduce the class of two point boundary value descriptor systems tpbvds discrete time systems described by possibly linear dynamics and a set of boundary conditions constraining the values of the system state at the two endpoints of the system s interval of definition by introducing a standard form for regular pencils we obtain a new and simple generalized cayley hamilton theorem that simplifies our investigation of well posedness green s function solutions and reachability and observability for tpbvds there are two distinct notions of reachability and observability that one can define for tpbvds associated with processes that propagate inward from and outward toward the boundaries we investigate each of these in detail obtaining among other things far simpler forms for the reachability and observability results than found previously in the literature in addition we describe several methods for the efficient solution of tpbvds one involving recursions from each end of t

solutions to the output regulation problem of linear singular systems in this paper we revisit the output regulation problem for linear singular systems and identify an important case of the general regulation problem where the measurement output is identical to the vector to be regulated we derive a necessary and sufficient condition for the regulation problem to be solvable via either full information feedback or error feedback then we show how full information feedback and error feedback controllers can be constructed explicitly

a study on relationship between generalization abilities and fuzziness of base classifiers in ensemble learning we investigate essential relationships between generalization capabilities and fuzziness of fuzzy classifiers viz the classifiers whose outputs are vectors of membership grades of a pattern to the individual classes the study makes a claim and offers sound evidence behind the observation that higher fuzziness of a fuzzy classifier may imply better generalization aspects of the classifier especially for classification data exhibiting complex boundaries this observation is not intuitive with a commonly accepted position in traditional pattern recognition the relationship that obeys the conditional maximum entropy principle is experimentally confirmed furthermore the relationship can be explained by the fact that samples located close to classification boundaries are more difficult to be correctly classified than the samples positioned far from the boundaries this relationship is expected to provide some guidelines as to the improvement of generalization aspects of fuzzy classifiers

an analysis of ensemble pruning techniques based on ordered aggregation several pruning strategies that can be used to reduce the size and increase the accuracy of bagging ensembles are analyzed these heuristics select subsets of complementary classifiers that when combined can perform better than the whole ensemble the pruning methods investigated are based on modifying the order of aggregation of classifiers in the ensemble in the original bagging algorithm the order of aggregation is left unspecified when this order is random the generalization error typically decreases as the number of classifiers in the ensemble increases if an appropriate ordering for the aggregation process is devised the generalization error reaches a minimum at intermediate numbers of classifiers this minimum lies below the asymptotic error of bagging pruned ensembles are obtained by retaining a fraction of the classifiers in the ordered ensemble the performance of these pruned ensembles is evaluated in several benchmark classification tasks under different training conditions the results of this empirical investigation show that ordered aggregation can be used for the efficient generation of pruned ensembles that are competitive in terms of performance and robustness of classification with computationally more costly methods that directly select optimal or near optimal subensembles

exact 1 norm support vector machines via unconstrained convex differentiable minimization support vector machines utilizing the 1 norm typically set up as linear programs mangasarian 2000 bradley and mangasarian 1998 are formulated here as a completely unconstrained minimization of a convex differentiable piecewise quadratic objective function in the dual space the objective function which has a lipschitz continuous gradient and contains only one additional finite parameter can be minimized by a generalized newton method and leads to an exact solution of the support vector machine problem the approach here is based on a formulation of a very general linear program as an unconstrained minimization problem and its application to support vector machine classification problems the present approach which generalizes both mangasarian 2004 and fung and mangasarian 2004 is also applied to nonlinear approximation where a minimal number of nonlinear kernel functions are utilized to approximate a function from a given number of function values

the generalized eigenstructure problem in linear system theory the algebraic theory of linear time invariant systems has been studied in large detail during the past few decades and numerous computational algorithms have been developed to solve problems arising in this context in this paper the numerical aspects of a certain class of such algorithms dealing with what the author calls generalized eigenstructure problems are discussed some new and or modified algorithms are presented both the nmnerical stability of the algorithms and the conditioning of the problems they solve are analyzed using numerical criteria

fuzzy nonlinear regression analysis using a random weight network modeling a fuzzy in fuzzy out system where both inputs and outputs are uncertain is of practical and theoretical importance fuzzy nonlinear regression fnr is one of the approaches used most widely to model such systems in this study we propose the use of a random weight network rwn to develop a fnr model called fnrrwn where both the inputs and outputs are triangular fuzzy numbers unlike existing fnr models based on back propagation bp and radial basis function rbf networks fnrrwn does not require iterative adjustment of the network weights and biases instead the input layer weights and hidden layer biases of fnrrwn are selected randomly the output layer weights for fnrrwn are calculated analytically based on a derived updating rule which aims to minimize the integrated squared error between cut sets that correspond to the predicted fuzzy outputs and target fuzzy outputs respectively in fnrrwn the integrated squared error is solved approximately by riemann integral theory the experimental results show that the proposed fnrrwn method can effectively approximate a fuzzy in fuzzy out system fnrrwn obtains better prediction accuracy in a lower computational time compared with existing fnr models based on bp and rbf networks

a probabilistic model of classifier competence for dynamic ensemble selection the concept of a classifier competence is fundamental to multiple classifier systems mcss in this study a method for calculating the classifier competence is developed using a probabilistic model in the method first a randomised reference classifier rrc whose class supports are realisations of the random variables with beta probability distributions is constructed the parameters of the distributions are chosen in such a way that for each feature vector in a validation set the expected values of the class supports produced by the rrc and the class supports produced by a modelled classifier are equal this allows for using the probability of correct classification of the rrc as the competence of the modelled classifier the competences calculated for a validation set are then generalised to an entire feature space by constructing a competence function based on a potential function model or regression three systems based on a dynamic classifier selection and a dynamic ensemble selection des were constructed using the method developed the des based system had statistically significant higher average rank than the ones of eight benchmark mcss for 22 data sets and a heterogeneous ensemble the results obtained indicate that the full vector of class supports should be used for evaluating the classifier competence as this potentially improves performance of mcss

eigenclassifiers for combining correlated classifiers in practice classifiers in an ensemble are not independent this paper is the continuation of our previous work on ensemble subset selection a ulas m semerci o t yildiz e alpaydin incremental construction of classifier and discriminant ensembles information sciences 179 9 2009 1298 1318 and has two parts first we investigate the effect of four factors on correlation i algorithms used for training ii hyperparameters of the algorithms iii resampled training sets iv input feature subsets simulations using 14 classifiers on 38 data sets indicate that hyperparameters and overlapping training sets have higher effect on positive correlation than features and algorithms second we propose postprocessing before fusing using principal component analysis pca to form uncorrelated eigenclassifiers from a set of correlated experts combining the information from all classifiers may be better than subset selection where some base classifiers are pruned before combination because using all allows redundancy

remarks on controllability of implicit linear discrete time systems abstract some results concerning controllability of implicit linear discrete time system of the form ex k 1 fx k gu k are given it is not assumed that the corresponding pencil se f is regular the concept of x i x f controllability is defined and it is shown that it generalizes various definitions of controllability used in the existing literature of the subject computable geometric conditions for an implicit system to be x 1 x f controllable are given some particular cases are discussed the existence of a universal time of controllability is also proven

greedy regression ensemble selection theory and an application to water quality prediction this paper studies the greedy ensemble selection family of algorithms for ensembles of regression models these algorithms search for the globally best subset of regressors by making local greedy decisions for changing the current subset we abstract the key points of the greedy ensemble selection algorithms and present a general framework which is applied to an application domain with important social and commercial value water quality prediction

reachability and controllability of discrete time descriptor systems the causal and non causal reachability grammians are used in order to define the reachability subspace of discrete time systems controllability of the discrete time regular descriptor system by means of the forward solution is also examined and new criteria for testing reachability controllability from the origin and controllability to the origin in the forward sense are proved

weighted classifier ensemble based on quadratic form diversity and accuracy are the two key factors that decide the ensemble generalization error constructing a good ensemble method by balancing these two factors is difficult because increasing diversity is at the cost of reducing accuracy normally in order to improve the performance of an ensemble while avoiding the difficulty derived of balancing diversity and accuracy we propose a novel method that weights each classifier in the ensemble by maximizing three different quadratic forms in this paper the optimal weight of individual classifiers is obtained by minimizing the ensemble error rather than analyzing diversity and accuracy since it is difficult to minimize the general form of the ensemble error directly we approximate the error in an objective function subject to two constraints w i 1 and 1 w i 1 particularly we introduce an error term with a weight vector w0 and subtract this error with the quadratic form to obtain our approximated error this subtraction makes minimizing the approximation form equivalent to maximizing the original quadratic form theoretical analysis finds that when the value of the quadratic form is maximized the error of an ensemble system with the corresponding optimal weight w will be smallest especially compared with the ensemble with w0 finally we demonstrate improved classification performance from the experimental results of an artificial dataset uci datasets and polsar image data a new weighted classifier ensemble method is proposed an approximation form of the ensemble error is introduced an optimal weight vector is sought based on minimizing the approximation form it is converted into maximizing quadratic forms by invoking a known weight vector the larger the value of the quadratic form is the lower the ensemble error is

active sample selection based incremental algorithm for attribute reduction with rough sets attribute reduction with rough sets is an effective technique for obtaining a compact and informative attribute set from a given dataset however traditional algorithms have no explicit provision for handling dynamic datasets where data present themselves in successive samples incremental algorithms for attribute reduction with rough sets have been recently introduced to handle dynamic datasets with large samples though they have high complexity in time and space to address the time space complexity issue of the algorithms this paper presents a novel incremental algorithm for attribute reduction with rough sets based on the adoption of an active sample selection process and an insight into the attribute reduction process this algorithm first decides whether each incoming sample is useful with respect to the current dataset by the active sample selection process a useless sample is discarded while a useful sample is selected to update a reduct at the arrival of a useful sample the attribute reduction process is then employed to guide how to add and or delete attributes in the current reduct the two processes thus constitute the theoretical framework of our algorithm the proposed algorithm is finally experimentally shown to be efficient in time and space

fuzzy neural networks a survey abstract in this paper a fuzzy neural network will be a layered feedforward neural net that has fuzzy signals and or fuzzy weights we survey recent results on learning algorithms and applications for fuzzy neural networks