using extension sets to aggregate partial rankings in a flexible setting this paper deals with the rank aggregation problem in a general setting in particular we approach the problem for any kind of ranking complete or incomplete and with or without ties the underlying idea behind our approach is to take into account the so called extension set of a ranking that is the set of permutations that are compatible with the given ranking in this way we aim to manage the uncertainty inherent to this problem when not all the items are ranked and or when some items are equally preferred we develop two approaches a general one based on this idea and a constrained version in which the extension set is limited by not allowing non ranked items to be placed in an existent bucket of tied items we test our proposal by coupling it with two different algorithms we formalize our approaches mathematically and also carry out an extensive experimental evaluation by using 22 datasets the results show that the use of our extension sets based approaches to compute the precedence matrix clearly outperforms the standard way of computing the preference matrix by only using the information explicitly provided by the rankings in the sample

frsvc towards making support vector clustering consume less abstract in spite of with great advantage of discovering arbitrary shapes of clusters support vector clustering svc is frustrated by large scale data especially on resource limited platform it is due to pricey storage and computation consumptions from solving dual problem and labeling clusters upon the pre computed kernel matrix and sampling point pairs respectively towards on it we first present a dual coordinate descent method to reformulate the solver that leads to a flexible training phase carried out on any runtime platform with without sufficient memory then a novel labeling phase who does connectivity analysis between two nearest neighboring decomposed convex hulls referring to clusters is proposed in which a new designed strategy namely sample once connected checking first tries to reduces the scope of sampling analysis by integrating them together a faster and reformulated svc frsvc is created with less consumption achieved according to comparative analysis of time and space complexities furthermore experimental results confirm a significant improvement on flexibility of selective efficiency without losing accuracy with which a balance can be easily reached on the basis of resources a platform equipped

swarm intelligence and evolutionary algorithms the popularity of metaheuristics especially swarm intelligence and evolutionary algorithms has increased rapidly over the last two decades numerous algorithms are proposed each year and progressively more novel applications are being found however different metaheuristics are often compared by their performance on problems with an arbitrarily fixed number of allowed function calls there are surprisingly few papers that explore the relationship between the relative performance of numerous metaheuristics on versatile numerical real world problems and the number of allowed function calls in this study the performance of 33 various metaheuristics proposed between 1960 and 2016 have been tested on 22 numerical real world problems from different fields of science with the maximum number of function calls varying between 5000 and 500 000 it is confirmed that the algorithms that succeed in comparisons when the computational budget is low are among the poorest performers when the computational budget is high and vice versa among the tested variants particle swarm optimization algorithms and some new types of metaheuristics perform relatively better when the number of allowed function calls is low whereas differential evolution and genetic algorithms perform better relative to other algorithms when the computational budget is large it is difficult to find any metaheuristic that would perform adequately over all of the numbers of function calls tested it was also found that some algorithms may become completely unreliable on specific real world problems even though they perform reasonably on others

on the 2 tuples based genetic tuning performance for fuzzy rule based classification systems in imbalanced data sets when performing a classification task we may find some data sets with a different class distribution among their patterns this problem is known as classification with imbalanced data sets and it appears in many real application areas for this reason it has recently become a relevant topic in the area of machine learning the aim of this work is to improve the behaviour of fuzzy rule based classification systems frbcss in the framework of imbalanced data sets by means of a tuning step specifically we adapt the 2 tuples based genetic tuning approach to classification problems showing the good synergy between this method and some frbcss our empirical results show that the 2 tuples based genetic tuning increases the performance of frbcss in all types of imbalanced data furthermore when the initial rule base built by a fuzzy rule learning methodology obtains a good behaviour in terms of accuracy we achieve a higher improvement in performance for the whole model when applying the genetic 2 tuples post processing step this enhancement is also obtained in the case of cooperation with a preprocessing stage proving the necessity of rebalancing the training set before the learning phase when dealing with imbalanced data

a sharper bonferroni procedure for multiple tests of significance summary a simple procedure for multiple tests of significance based on individual p values is derived this simple procedure is sharper than holm s 1979 sequentially rejective procedure both procedures contrast the ordered p values with the same set of critical values holm s procedure rejects an hypothesis only if its p value and each of the smaller p values are less than their corresponding critical values the new procedure rejects all hypotheses with smaller or equal p values to that of any one found less than its critical value

drcw ovo one vs one strategy is a common and established technique in machine learning to deal with multi class classification problems it consists of dividing the original multi class problem into easier to solve binary subproblems considering each possible pair of classes since several classifiers are learned their combination becomes crucial in order to predict the class of new instances due to the division procedure a series of difficulties emerge at this stage such as the non competence problem each classifier is learned using only the instances of its corresponding pair of classes and hence it is not competent to classify instances belonging to the rest of the classes nevertheless at classification time all the outputs of the classifiers are taken into account because the competence cannot be known a priori the classification problem would be solved on this account we develop a distance based combination strategy which weights the competence of the outputs of the base classifiers depending on the closeness of the query instance to each one of the classes our aim is to reduce the effect of the non competent classifiers enhancing the results obtained by the state of the art combinations for one vs one strategy we carry out a thorough experimental study supported by the proper statistical analysis showing that the results obtained by the proposed method outperform both in terms of accuracy and kappa measures the previous combinations for one vs one strategy highlightsthe non competence is an important problem in one vs one strategy we develop a distance based combination strategy based on dynamic classifier weighting strategies weights are settled depending on the closeness of the test instance to each one of the classesthe effect of the non competent classifiers is reduced the new strategy enhances the results obtained w r t the state of the art aggregations

particle swarm optimization with adaptive mutation for multimodal optimization particle swarm optimization pso is a population based stochastic search algorithm which has shown a good performance over many benchmark and real world optimization problem like other stochastic algorithms pso also easily falls into local optima in solving complex multimodal problems to help trapped particles escape from local minima this paper presents a new pso variant called ampso by employing an adaptive mutation strategy to verify the performance of ampso a set of well known complex multimodal benchmarks are used in the experiments simulation results demonstrate that the proposed mutation strategy can efficiently improve the performance of pso

integration of an index to preserve the semantic interpretability in the multiobjective evolutionary rule selection and tuning of linguistic fuzzy systems in this paper we propose an index that helps preserve the semantic interpretability of linguistic fuzzy models while a tuning of the membership functions mfs is performed the proposed index is the aggregation of three metrics that preserve the original meanings of the mfs as much as possible while a tuning of their definition parameters is performed additionally rule selection mechanisms can be used to reduce the model complexity which involves another important interpretability aspect to this end we propose a postprocessing multiobjective evolutionary algorithm that performs rule selection and tuning of fuzzy rule based systems with three objectives accuracy semantic interpretability maximization and complexity minimization we tested our approach on nine real world regression datasets in order to analyze the interaction between the fuzzy rule selection approach and the tuning approach these are also individually proved in a multiobjective framework and compared with their respective single objective counterparts we compared the different approaches by applying nonparametric statistical tests for pairwise and multiple comparisons taking into consideration three representative points from the obtained pareto fronts in the case of the multiobjective based approaches results confirm the effectiveness of our approach and a wide range of solutions is obtained which are not only more interpretable but are also more accurate

mandatory leaf node prediction in hierarchical multilabel classification in hierarchical classification the output labels reside on a tree or directed acyclic graph dag structured hierarchy on testing the prediction paths of a given test example may be required to end at leaf nodes of the label hierarchy this is called mandatory leaf node prediction mlnp and is particularly useful when the leaf nodes have much stronger semantic meaning than the internal nodes however while there have been a lot of mlnp methods in hierarchical multiclass classification performing mlnp in hierarchical multilabel classification is difficult in this paper we propose novel mlnp algorithms that consider the global label hierarchy structure we show that the joint posterior probability over all the node labels can be efficiently maximized by dynamic programming for label trees or greedy algorithm for label dags in addition both algorithms can be further extended for the minimization of the expected symmetric loss experiments are performed on real world mlnp data sets with label trees and label dags the proposed method consistently outperforms other hierarchical and flat multilabel classification methods

enhancing particle swarm optimization using generalized opposition based learning particle swarm optimization pso has been shown to yield good performance for solving various optimization problems however it tends to suffer from premature convergence when solving complex problems this paper presents an enhanced pso algorithm called gopso which employs generalized opposition based learning gobl and cauchy mutation to overcome this problem gobl can provide a faster convergence and the cauchy mutation with a long tail helps trapped particles escape from local optima the proposed approach uses a similar scheme as opposition based differential evolution ode with opposition based population initialization and generation jumping using gobl experiments are conducted on a comprehensive set of benchmark functions including rotated multimodal problems and shifted large scale problems the results show that gopso obtains promising performance on a majority of the test problems

an improved bonferroni procedure for multiple tests of significance summary a modification of the bonferroni procedure for testing multiple hypotheses is presented the method based on the ordered p values of the individual tests is less conservative than the classical bonferroni procedure but is still simple to apply a simulation study shows that the probability of a type i error of the procedure does not exceed the nominal significance level a for a variety of multivariate normal and multivariate gamma test statistics for independent tests the procedure has type i error probability equal to a the method appears particularly advantageous over the classical bonferroni procedure when several highly correlated test statistics are involved

modified sequentially rejective multiple test procedures abstract suppose that n hypotheses h 1 h 2 h n with associated test statistics t 1 t 2 t n are to be tested by a procedure with experimentwise significance level the probability of rejecting one or more true hypotheses smaller than or equal to some specified value a commonly used procedure satisfying this condition is the bonferroni b procedure which consists of rejecting h i for any i iff the associated test statistic t i is significant at the level n holm 1979 introduced a modified bonferroni procedure with greater power than the b procedure under holm s sequentially rejective bonferroni srb procedure if any hypothesis is rejected at the level n the denominator of for the next test is n 1 and the criterion continues to be modified in a stagewise manner with the denominator of reduced by 1 each time a hypothesis is rejected so that tests can be conducted at successively higher significance levels holm proved that the experimentwise significance level

some metaheuristics should be simplified abstract users have a large and constantly increasing number of optimization metaheuristics at their disposal in pursuit of ever better approaches popular and successful methods are often improved a number of times in a row or various methods are hybridized one should however pay attention to whether such deliberately improved or hybridized methods do not become unnecessarily complicated or if some of their elements do not introduce a structural bias in the first case the algorithm should be simplified by eliminating the unneeded elements this will make it more clear to the users and as shown in this study may even improve the results in the second case operators that are responsible for over frequent sampling of some part of the search space have to be removed in this study we address the problem of over complexity of metaheuristics focusing on two joint winners of the cec2016 competition on single objective numerical optimization l shade epsin and umoea ii algorithms it is shown that each of them includes a procedure which introduces a structural bias by attracting population towards the origin o as discussed in the text in the case of seven out of 30 benchmark problems considered in the cec2016 competition the objective function values in the origin o are better than those found by the vast majority of algorithms hence structural bias affects the results obtained by l shade epsin and umoea ii in this work we simplify both algorithms by removing operators that are the main cause of structural bias further we show that in the l shade epsin algorithm the ensemble sinusoidal adaptation mechanism of control parameter f that is used during the first half of the search is not needed slightly better results on both artificial benchmarks used in the cec2016 competition and on a wide set of real world problems may be obtained if during that period of the search the value of f is simply set to 0 5 such a simplified algorithm is competitive against a large number of metaheuristics and may be much easier for the users to understand

a general framework for the statistical analysis of the sources of variance for classification error estimators estimating the prediction error of classifiers induced by supervised learning algorithms is important not only to predict its future error but also to choose a classifier from a given set model selection if the goal is to estimate the prediction error of a particular classifier the desired estimator should have low bias and low variance however if the goal is the model selection in order to make fair comparisons the chosen estimator should have low variance assuming that the bias term is independent from the considered classifier this paper follows the analysis proposed in 1 about the statistical properties of k fold cross validation estimators and extends it to the most popular error estimators resubstitution holdout repeated holdout simple bootstrap and 0 632 bootstrap estimators without and with stratification we present a general framework to analyze the decomposition of the variance of different error estimators considering the nature of the variance irreducible reducible variance and the different sources of sensitivity internal external sensitivity an extensive empirical study has been performed for the previously mentioned estimators with naive bayes and c4 5 classifiers over training sets obtained from assorted probability distributions the empirical analysis consists of decomposing the variances following the proposed framework and checking the independence assumption between the bias and the considered classifier based on the obtained results we propose the most appropriate error estimations for model selection under different experimental conditions

a stagewise rejective multiple test procedure based on a modified bonferroni test summary simes 1986 has proposed a modified bonferroni procedure for the test of an overall hypothesis which is the combination of n individual hypotheses in contrast to the classical bonferroni procedure it is not obvious how statements about individual hypotheses are to be made for this procedure in the present paper a multiple test procedure allowing statements on individual hypotheses is proposed it is based on the principle of closed test procedures marcus peritz gabriel 1976 and controls the multiple level a

progressive subspace ensemble learning there are not many classifier ensemble approaches which investigate the data sample space and the feature space at the same time and this multi pronged approach will be helpful for constructing more powerful learning models for example the adaboost approach only investigates the data sample space while the random subspace technique only focuses on the feature space to address this limitation we propose the progressive subspace ensemble learning approach psel which takes into account the data sample space and the feature space at the same time specifically psel first adopts the random subspace technique to generate a set of subspaces then a progressive selection process based on new cost functions that incorporate current and long term information to select the classifiers sequentially will be introduced finally a weighted voting scheme is used to summarize the predicted labels and obtain the final result we also adopt a number of non parametric tests to compare psel and its competitors over multiple datasets the results of the experiments show that psel works well on most of the real datasets and outperforms a number of state of the art classifier ensemble approaches highlightsprogressive subspace ensemble learning it takes into account the data sample space and the feature space at the same time a progressive selection process based on new cost functions that incorporate current and long term information to select the classifiers sequentially will be introduced

effective lazy learning algorithm based on a data gravitation model for multi label learning in the last decade an increasing number of real world problems surrounding multi label data have appeared and multi label learning has become an important area of research the data gravitation model is an approach that applies the principles of the universal law of gravitation to resolve machine learning problems one advantage of the data gravitation model compared with other techniques is that it is based on simple principles with high performance levels this paper presents a multi label lazy algorithm based on a data gravitation model named mldgc mldgc directly handles multi label data and considers each instance as an atomic data particle the proposed multi label lazy algorithm was evaluated and compared to several state of the art multi label lazy methods on 34 datasets the results showed that our proposal outperformed state of the art lazy methods the experimental results were validated using non parametric statistical tests confirming the effectiveness of this data gravitation model for multi label lazy learning

epistatic arithmetic crossover based on cartesian graph product in ensemble differential evolution epistasis in genetics denotes an impact of one gene on the expression of the another genes this means that so called epistatic gene influences the characteristics of the so called hypostatic genes as a matter of fact there is no one to one correspondence between genes and traits in nature on the other hand values of offspring genes are inherited from the parents genes in this paper the impact of epistatic genes in evolutionary computation is studied where each epistatic gene in offspring depends on the corresponding hypostatic genes of its parents by an epistatic arithmetic crossover used in differential evolution thus epistatic genes are determined by the cartesian graph product of both parents presented as linear graphs the epistatic arithmetic crossover is applied as a mutation strategy to the ensemble differential evolution the results of extensive experiments conducted on cec 14 function benchmark suite showed a great potential of the proposed algorithm and encouraged us to start to experiment with other graph products as well

regarding the rankings of optimization heuristics based on artificially constructed benchmark functions novel evolutionary algorithms are usually tested on sets of artificially constructed benchmark problems such problems are often created to make the search of one global extremum usually minimum tricky in this paper it is shown that benchmarking heuristics on either minimization or maximization of the same set of artificially created functions with equal bounds and number of allowed function calls may lead to very different ranking of tested algorithms as evolutionary algorithms and other heuristic optimizers are developed in order to be applicable to real world problems such result may raise doubts on the practical meaning of benchmarking them on artificial functions as there is little reason that searching for the minimum of such functions should be more important than searching for their maximum thirty optimization heuristics including a number of variants of differential evolution as well as other kinds of evolutionary algorithms particle swarm optimization direct search methods and following the idea borrowed from no free lunch pure random search are tested in the paper some discussion regarding the choice of the mean or the median performance for comparison is addressed and a short debate on the overall performance of particular methods is given

a symmetry based multiobjective clustering technique for automatic evolution of clusters in this paper the problem of automatic clustering a data set is posed as solving a multiobjective optimization moo problem optimizing a set of cluster validity indices simultaneously the proposed multiobjective clustering technique utilizes a recently developed simulated annealing based multiobjective optimization method as the underlying optimization strategy here variable number of cluster centers is encoded in the string the number of clusters present in different strings varies over a range the points are assigned to different clusters based on the newly developed point symmetry based distance rather than the existing euclidean distance two cluster validity indices one based on the euclidean distance xb index and another recently developed point symmetry distance based cluster validity index sym index are optimized simultaneously in order to determine the appropriate number of clusters present in a data set thus the proposed clustering technique is able to detect both the proper number of clusters and the appropriate partitioning from data sets either having hyperspherical clusters or having point symmetric clusters a new semi supervised method is also proposed in the present paper to select a single solution from the final pareto optimal front of the proposed multiobjective clustering technique the efficacy of the proposed algorithm is shown for seven artificial data sets and six real life data sets of varying complexities results are also compared with those obtained by another multiobjective clustering technique mock two single objective genetic algorithm based automatic clustering techniques vgaps clustering and gcuk clustering

multi task tsk fuzzy system modeling using inter task correlation information the classical fuzzy system modeling methods have been typically developed for the single task modeling scene which is essentially not in accordance with many practical applications where a multi task problem must be considered for the given modeling task although a multi task problem can be decomposed into many single task sub problems the modeling results indeed tell us that the individual modeling approach will not be very suitable for multi task problems due to the ignorance of the inter task latent correlation between different tasks in order to circumvent this shortcoming a multi task takagi sugeno kang fuzzy system model is proposed based on the classical l2 norm takagi sugeno kang fuzzy system in this paper the proposed model cannot only take advantage of independent information of each task but also make use of the inter task latent correlation information effectively resulting to obtain better generalization performance for the built fuzzy systems experiments on synthetic and real world datasets demonstrate the applicability and distinctive performance of the proposed multi task fuzzy system model in multi task modeling scenarios

the weighted independent domination problem integer linear programming models and metaheuristic approaches abstract this work deals with the so called weighted independent domination problem which is an np hard combinatorial optimization problem in graphs in contrast to previous work this paper considers the problem from a non theoretical perspective the first contribution consists in the development of three integer linear programming models second two greedy heuristics are proposed finally the last contribution is a population based iterated greedy metaheuristic which is applied in two different ways 1 the metaheuristic is applied directly to each problem instance and 2 the metaheuristic is applied at each iteration of a higher level framework known as construct merge solve and adapt to sub instances of the tackled problem instances the results of the considered algorithmic approaches show that integer linear programming approaches can only compete with the developed metaheuristics in the context of graphs with up to 100 nodes when larger graphs are concerned the application of the populated based iterated greedy algorithm within the higher level framework works generally best the experimental evaluation considers graphs of different types sizes densities and ways of generating the node and edge weights

comparison and design of interpretable linguistic vs scatter frbss gm3m generalization and new rule meaning index for global assessment and local pseudo linguistic representation this work is devoted to defining more general interpretability indexes to be applied to any scatter or linguistic model implemented by any type of membership functions they are based on metrics that should take into account the semantic and inference issues the semantic issue in order to preserve the meaning of the linguistic labels and the inference issue since this can influence the behavior of the rules on the other hand these metrics have been designed to be intuitive in order to support the analysis or selection of a final model and to favor a low computational cost within an optimization process in order to check their usefulness a multi objective evolutionary algorithm simultaneously performing a rule selection and an adjustment of the fuzzy partitions is guided by the proposed indexes on several benchmark data sets to obtain models with different degrees of accuracy and interpretability in addition using these metrics a local analysis can be carried out between models of a different nature this local analysis through the model components gives support to the user to make the best choice from among the models

parametric methods for comparing the performance of two classification algorithms evaluated by k fold cross validation on multiple data sets a popular procedure for identifying which one of two classification algorithms has a better performance is to test them on multiple data sets and the accuracies resulting from k fold cross validation are aggregated to draw a conclusion several nonparametric methods have been proposed for this purpose while parametric methods will be a better choice to determine the superior algorithm when the assumptions for deriving sampling distributions can be satisfied in this paper we consider every accuracy estimate resulting from the instances in a fold or a data set as a point estimator instead of a fixed value to derive the sampling distribution of the point estimator for comparing the performance of two classification algorithms the test statistics for both data set and fold averaging levels are proposed and the ways to calculate their degrees of freedom are also presented twelve data sets are chosen to demonstrate that our parametric methods can be used to effectively compare the performance of two classification algorithms on multiple data sets several critical issues in using our parametric methods and the nonparametric ones proposed in a previous study are then discussed parametric methods for performance comparison of two classification algorithms are proposed both independent and matched samples are considered the experimental results on twelve data sets demonstrate the effectiveness of our parametric methods

ifs coco instance and feature selection based on cooperative coevolution with nearest neighbor rule feature and instance selection are two effective data reduction processes which can be applied to classification tasks obtaining promising results although both processes are defined separately it is possible to apply them simultaneously this paper proposes an evolutionary model to perform feature and instance selection in nearest neighbor classification it is based on cooperative coevolution which has been applied to many computational problems with great success the proposed approach is compared with a wide range of evolutionary feature and instance selection methods for classification the results contrasted through non parametric statistical tests show that our model outperforms previously proposed evolutionary approaches for performing data reduction processes in combination with the nearest neighbor rule

a robust wrap reduction algorithm for fringe projection profilometry and applications in magnetic resonance imaging in this paper we present an effective algorithm to reduce the number of wraps in a 2d phase signal provided as input the technique is based on an accurate estimate of the fundamental frequency of a 2d complex signal with the phase given by the input and the removal of a dependent additive term from the phase map unlike existing methods based on the discrete fourier transform dft the frequency is computed by using noise robust estimates that are not restricted to integer values then to deal with the problem of a non integer shift in the frequency domain an equivalent operation is carried out on the original phase signal this consists of the subtraction of a tilted plane whose slope is computed from the frequency followed by a re wrapping operation the technique has been exhaustively tested on fringe projection profilometry fpp and magnetic resonance imaging mri signals in addition the performance of several frequency estimation methods has been compared the proposed methodology is particularly effective on fpp signals showing a higher performance than the state of the art wrap reduction approaches in this context it contributes to canceling the carrier effect at the same time as it eliminates any potential slope that affects the entire signal its effectiveness on other carrier free phase signals e g mri is limited to the case that inherent slopes are present in the phase data

positional and confidence voting based consensus functions for fuzzy cluster ensembles consensus clustering i e the task of combining the outcomes of several clustering systems into a single partition has lately attracted the attention of researchers in the unsupervised classification field as it allows the creation of clustering committees that can be applied with multiple interesting purposes such as knowledge reuse or distributed clustering however little attention has been paid to the development of algorithms known as consensus functions especially designed for consolidating the outcomes of multiple fuzzy or soft clustering systems into a single fuzzy partition despite the fact that fuzzy clustering is far more informative than its crisp counterpart as it provides information regarding the degree of association between objects and clusters that can be helpful for deriving richer descriptive data models for this reason this paper presents a set of fuzzy consensus functions capable of creating soft consensus partitions by fusing a collection of fuzzy clusterings our proposals base clustering combination on a cluster disambiguation process followed by the application of positional and confidence voting techniques the modular design of these algorithms makes it possible to sequence their constituting steps in different manners which allows to derive versions of the proposed consensus functions optimized from a computational standpoint the proposed consensus functions have been evaluated in terms of the quality of the consensus partitions they deliver and in terms of their running time on multiple benchmark data sets a comparison against several representative state of the art consensus functions reveals that our proposals constitute an appealing alternative for conducting fuzzy consensus clustering as they are capable of yielding high quality consensus partitions at a low computational cost

classifying imbalanced data sets using similarity based hierarchical decomposition classification of data is difficult if the data is imbalanced and classes are overlapping in recent years more research has started to focus on classification of imbalanced data since real world data is often skewed traditional methods are more successful with classifying the class that has the most samples majority class compared to the other classes minority classes for the classification of imbalanced data sets different methods are available although each has some advantages and shortcomings in this study we propose a new hierarchical decomposition method for imbalanced data sets which is different from previously proposed solutions to the class imbalance problem additionally it does not require any data pre processing step as many other solutions need the new method is based on clustering and outlier detection the hierarchy is constructed using the similarity of labeled data subsets at each level of the hierarchy with different levels being built by different data and feature subsets clustering is used to partition the data while outlier detection is utilized to detect minority class samples the comparison of the proposed method with state of art the methods using 20 public imbalanced data sets and 181 synthetic data sets showed that the proposed method s classification performance is better than the state of art methods it is especially successful if the minority class is sparser than the majority class it has accurate performance even when classes have sub varieties and minority and majority classes are overlapping moreover its performance is also good when the class imbalance ratio is low i e classes are more imbalanced a novel method for imbalanced dataset classification a new hierarchical classifier which does not use a fixed feature class hierarchy uses clustering and outlier detection to construct the hierarchy shows that different feature spaces can be used to build a hierarchy successful when the class imbalanced ratio is low classes are highly overlapping

cost conscious comparison of supervised learning algorithms over multiple data sets in the literature there exist statistical tests to compare supervised learning algorithms on multiple data sets in terms of accuracy but they do not always generate an ordering we propose multi 2test a generalization of our previous work for ordering multiple learning algorithms on multiple data sets from best to worst where our goodness measure is composed of a prior cost term additional to generalization error our simulations show that multi 2test generates orderings using pairwise tests on error and different types of cost using time and space complexity of the learning algorithms

adjusted p values for simultaneous inference this paper proposes that results from simultaneous tests be reported as adjusted p values such that if the adjusted p value for an individual hypothesis is less than the chosen significance level of a then the hypothesis is rejected with an experimentwise error rate of no more than ax examples are given of adjusted p values for multiple comparisons in the analysis of variance and of adjusted p values based on the bonferroni procedure and modifications of that procedure by holm 1979 scandinavian journal of statistics 6 65 70 hochberg 1988 biometrika 75 800 802 and hommel 1988 biometrika 75 383 386 the modified bonferroni based procedures are much more powerful than the original bonferroni procedure and they deserve wider use in addition to the above a procedure is outlined for obtaining adjusted p values for any closed test procedure

an extensive comparative study of cluster validity indices the validation of the results obtained by clustering algorithms is a fundamental part of the clustering process the most used approaches for cluster validation are based on internal cluster validity indices although many indices have been proposed there is no recent extensive comparative study of their performance in this paper we show the results of an experimental work that compares 30 cluster validity indices in many different environments with different characteristics these results can serve as a guideline for selecting the most suitable index for each possible application and provide a deep insight into the performance differences between the currently available indices

multilabel classifiers with a probabilistic thresholding strategy in multilabel classification tasks the aim is to find hypotheses able to predict for each instance a set of classes or labels rather than a single one some state of the art multilabel learners use a thresholding strategy which consists in computing a score for each label and then predicting the set of labels whose score is higher than a given threshold when this score is the estimated posterior probability the selected threshold is typically 0 5 in this paper we introduce a family of thresholding strategies which take into account the posterior probability of all possible labels to determine a different threshold for each instance thus we exploit some kind of interdependence among labels to compute this threshold which is optimal regarding a given expected loss function we found experimentally that these strategies outperform other thresholding options for multilabel classification they provide an efficient method to implement a learner which considers the interdependence among labels in the sense that the overall performance of the prediction of a set of labels prevails over that of each single label

on the stopping criteria for k nearest neighbor in positive unlabeled time series classification problems positive unlabeled time series classification has become an important area during the last decade as often vast amounts of unlabeled time series data are available but obtaining the corresponding labels is difficult in this situation positive unlabeled learning is a suitable option to mitigate the lack of labeled examples in particular self training is a widely used technique due to its simplicity and adaptability within this technique the stopping criterion i e the decision of when to stop labeling is a critical part especially in the positive unlabeled context we propose a self training method that follows the positive unlabeled approach for time series classification and a family of parameter free stopping criteria for this method our proposal uses a graphical analysis applied to the minimum distances obtained by the k nearest neighbor as the base learner to estimate the class boundary the proposed method is evaluated in an experimental study involving various time series classification datasets the results show that our method outperforms the transductive results obtained by previous models

hydr mi a hybrid algorithm to reduce dimensionality in multiple instance learning feature selection techniques have been successfully applied in many applications for making supervised learning more effective and efficient these techniques have been widely used and studied in traditional supervised learning settings where each instance is expected to have a label in multiple instance learning mil each example or bag consists of a variable set of instances and the label is known for the bag as a whole but not for the individual instances it consists of therefore utilizing these labels for feature selection in mil becomes less straightforward in this paper we study a new feature subset selection method for mil called hydr mi hybrid dimensionality reduction method for multiple instance learning the hybrid consists of the filter component based on an extension of the relieff algorithm developed for working with mil and the wrapper component based on a genetic algorithm that optimizes the search for the best feature subset from a reduced set of features output by the filter component we conducted an extensive experimental evaluation of our method on five benchmark datasets and 17 classification algorithms for mil the results of our study show the potential of the proposed hybrid with respect to the desirable effect it produces a significant improvement of the predictive performance of many mil classification techniques as compared to the effect of filter based feature selection this is achieved due to the possibility to decide how many of the top ranked features are useful for each particular algorithm and the possibility to discard redundant attributes

solving multi class problems with linguistic fuzzy rule based classification systems based on pairwise learning and preference relations this paper deals with multi class classification for linguistic fuzzy rule based classification systems the idea is to decompose the original data set into binary classification problems using the pairwise learning approach confronting all pair of classes and to obtain an independent fuzzy system for each one of them along the inference process each fuzzy rule based classification system generates an association degree for both of its corresponding classes and these values are encoded into a fuzzy preference relation our analysis is focused on the final step that returns the predicted class label specifically we propose to manage the fuzzy preference relation using a non dominance criterion on the different alternatives contrasting the behaviour of this model with both the classical weighted voting scheme and a decision rule that combines the fuzzy relations of preference conflict and ignorance by means of a voting strategy our experimental study is carried out using two different linguistic fuzzy rule learning methods for which we show that the non dominance criterion is a good alternative in comparison with the previously mentioned aggregation mechanisms this empirical analysis is supported through the corresponding statistical analysis using non parametrical tests

maximizing the area under the roc curve by pairwise feature combination the majority of the available classification systems focus on the minimization of the classification error rate this is not always a suitable metric specially when dealing with two class problems with skewed classes and cost distributions in this case an effective criterion to measure the quality of a decision rule is the area under the receiver operating characteristic curve auc that is also useful to measure the ranking quality of a classifier as required in many real applications in this paper we propose a nonparametric linear classifier based on the maximization of auc the approach lies on the analysis of the wilcoxon mann whitney statistic of each single feature and on an iterative pairwise coupling of the features for the optimization of the ranking of the combined feature by the pairwise feature evaluation the proposed procedure is essentially different from other classifiers using auc as a criterion experiments performed on synthetic and real data sets and comparisons with previous approaches confirm the effectiveness of the proposed method

the word problem in the baumslag group with a non elementary dehn function is polynomial time decidable abstract we prove that the word problem in the baumslag group g 1 2 a b a a b a 2 which has a non elementary dehn function is decidable in polynomial time

fingerprint indexing based on minutiae pairs and convex core point an efficient fingerprint indexing which uses ellipse properties is proposed a new fingerprint indexing is proposed using minutiae pairs and convex core point our fingerprint representation is very robust to distortions and noise the proposed fingerprint features are invariant to rotation and translation fingerprint identification is an important issue for identifying fingerprints and plays a key role in the fingerprint recognition systems however performing a fingerprint identification over a large database can be an inefficient task due to the lack of scalability and high computing times of fingerprint matching algorithms fingerprint indexing is a key strategy in automatic fingerprint identification systems afiss which allows us to reduce the number of candidates the search space and the occurrences of false acceptance in large databases in this paper an efficient indexing algorithm is proposed using minutiae pairs and convex core point which employs k means clustering and candidate list reduction criteria to improve the identification performance our proposal can effectively reduce the search space and number of candidates for fingerprint matching and thus achieves higher efficiency and significantly improves the system retrieval performance experimental results over some of the fingerprint verification competition fvc and the national institute of standards and technology nist databases prove the superiority of the proposed approach against some of the well known indexing algorithms

metsk hde in this contribution we propose a two stage method for accurate fuzzy modeling in high dimensional regression problems using approximate takagi sugeno kang fuzzy rule based systems in the first stage an evolutionary data base learning is performed involving variables granularities and slight fuzzy partition displacements together with an inductive rule base learning within the same process the second stage is a post processing process to perform a rule selection and a scatter based tuning of the membership functions for further refinement of the learned solutions moreover the second stage incorporates an efficient kalman filter to learn the coefficients of the consequent polynomial function in the takagi sugeno kang rules both stages include mechanisms that significantly improve the accuracy of the model and ensure a fast convergence in high dimensional and large scale regression datasets we tested our approach on 28 real world datasets with different numbers of variables and instances five well known methods have been executed as references we compared the different approaches by applying non parametric statistical tests for pair wise and multiple comparisons the results confirm the effectiveness of the proposed method showing better results in accuracy within a reasonable computing time

nmeef sd non dominated multiobjective evolutionary algorithm for extracting fuzzy rules in subgroup discovery a non dominated multiobjective evolutionary algorithm for extracting fuzzy rules in subgroup discovery nmeef sd is described and analyzed in this paper this algorithm which is based on the hybridization between fuzzy logic and genetic algorithms deals with subgroup discovery problems in order to extract novel and interpretable fuzzy rules of interest and the evolutionary fuzzy system nmeef sd is based on the well known non dominated sorting genetic algorithm ii nsga ii model but is oriented toward the subgroup discovery task using specific operators to promote the extraction of interpretable and high quality subgroup discovery rules the proposal includes different mechanisms to improve diversity in the population and permits the use of different combinations of quality measures in the evolutionary process an elaborate experimental study which was reinforced by the use of nonparametric tests was performed to verify the validity of the proposal and the proposal was compared with other subgroup discovery methods the results show that nmeef sd obtains the best results among several algorithms studied

analytical solution of a convection dispersion model with time dependent transport coefficients mathematical studies of solute transport in porous media have often utilized equivalent models of the transport process to remove undesired variability in the transport coefficients at the space and time scales of direct interest both deterministic and stochastic approaches in this genre produce an effective convection dispersion equation with time dependent coefficients this type of equation in one spatial dimension is investigated mathematically in the present paper a closed form solution of the solute transport equation is derived for a semi infinite spatial domain with arbitrary initial and boundary flux conditions it is shown that the solution reduces to well known results for special forms of the time dependent coefficients in general however a volterra integral equation of the second kind must be solved to evaluate the analytical solution of the transport equation we present a stable and convergent numerical scheme utilizing a trapezoidal quadrature rule for the solution of the volterra equation the method of solution developed should be applicable to a broad variety of solute transport problems including particularly those in heterogeneous porous media

infinitely imbalanced logistic regression in binary classification problems it is common for the two classes to be imbalanced one case is very rare compared to the other in this paper we consider the infinitely imbalanced case where one class has a finite sample size and the other class s sample size grows without bound for logistic regression the infinitely imbalanced case often has a useful solution under mild conditions the intercept diverges as expected but the rest of the coefficient vector approaches a non trivial and useful limit that limit can be expressed in terms of exponential tilting and is the minimum of a convex objective function the limiting form of logistic regression suggests a computational shortcut for fraud detection problems

semi supervised classification method through oversampling and common hidden space semi supervised classification methods attempt to improve classification performance based on a small amount of labeled data through full use of abundant unlabeled data although existing semi supervised classification methods have exhibited promising results in many applications they still have drawbacks including performance degeneration due to the introduction of unlabeled data and partially false labels in a small amount of labeled data to circumvent such drawbacks a new semi supervised classification method ochs ssc through oversampling and a common hidden space is proposed in the paper the primary characteristics of the proposed method include two aspects one is that unlabeled data are only used to generate new synthetic data to extend the minimal amount of labeled data the other is that the final classifier is learned in the extended feature space which is composed of the original feature space and the common hidden space found between labeled data and the synthetic data instead of the original feature space extensive experiments on 23 datasets indicate the effectiveness of the proposed method

fuzzy partitioning of continuous attributes through discretization methods to construct fuzzy decision tree classifiers abstract generating membership functions mfs from data is one of the fundamental challenges associated with the applications of fuzzy set theory this paper proposes a new two step algorithm which uses discretization methods for initial partitioning to generate mfs from data in the first step discretization algorithm divides domain of attributes to several partitions and then in the second step an mf is defined on each partition four different methods are proposed to define mfs in the second step the first method is based on partition width the second is based on standard deviation of examples the third is based on coverage rate of neighbor partitions npcr and the last one is based on coverage rate of partition pcr coverage rate of partition and coverage rate of neighbor partition are two new introduced parameters which can be used for mf generation in addition this paper proposes a new mf generation algorithm called fuzzy entropy based fuzzy partitioning febfp which is a specific version of the proposed two step algorithm with some modifications febfp uses fuzzy entropy of partition to generate mfs and involves the parameters of mfs in the process of mf generation to combine two steps of the algorithm non parametric statistical tests are used to compare fuzzy decision trees fdts induced using the mfs generated by the proposed methods employing different discretization algorithms as well as four mf generation methods experimental results show that eight methods outperform the others in terms of both accuracy and number of nodes among them trapezoidal mfs that are defined by pcr on partitions generated by zeta discretization algorithm outperform the others when the accuracy and complexity of fdt have the same degree of importance moreover the results show that the pcr and npcr mf definition methods perform better than the other ones

an extension on statistical comparisons of classifiers over multiple data sets for all pairwise comparisons in a recently published paper in jmlr dem sar 2006 recommends a set of non parametric statistical tests and procedures which can be safely used for comparing the performance of classifiers over multiple data sets after studying the paper we realize that the paper correctly introduces the basic procedures and some of the most advanced ones when comparing a control method however it does not deal with some advanced topics in depth regarding these topics we focus on more powerful proposals of statistical procedures for comparing n n classifiers moreover we illustrate an easy way of obtaining adjusted and comparable p values in multiple comparison procedures

weighted fuzzy rule interpolation based on ga based weight learning techniques in this paper we propose a weighted fuzzy interpolative reasoning method for sparse fuzzy rule based systems it is based on a genetic algorithm ga based weight learning technique the proposed method can deal with fuzzy rule interpolation with weighted antecedent variables it also can deal with fuzzy rule interpolation based on polygonal membership functions and bell shaped membership functions we also propose a ga based weight learning algorithm to automatically learn the optimal weights of the antecedent variables of the fuzzy rules furthermore we apply the proposed weighted fuzzy interpolative reasoning method and the proposed ga based weight learning algorithm to deal with the truck backer upper control problem the computer activity prediction problem multivariate regression problems and time series prediction problems based on statistical analysis techniques the experimental results show that the proposed weighted fuzzy interpolative reasoning method by the use of the optimally learned weights that were obtained by the proposed ga based weight learning algorithm has statistically significantly smaller error rates than the existing methods

a sequentially rejective test procedure based on a modified bonferroni inequality summary a sharper bonferroni procedure for multiple tests of significance is derived this procedure is an improvement of hochberg s 1988 procedure which contrasts the individual p values with corresponding critical points it is shown that hochberg s original procedure is conservative and can be made more powerful by enlarging the rejection region so that the type one error is exactly at the nominal level it is also shown that the modified procedure retains all the desired properties of the original procedure

anytime learning of decision trees the majority of existing algorithms for learning decision trees are greedy a tree is induced top down making locally optimal decisions at each node in most cases however the constructed tree is not globally optimal even the few non greedy learners cannot learn good trees when the concept is difficult furthermore they require a fixed amount of time and are not able to generate a better tree if additional time is available we introduce a framework for anytime induction of decision trees that overcomes these problems by trading computation speed for better tree quality our proposed family of algorithms employs a novel strategy for evaluating candidate splits a biased sampling of the space of consistent trees rooted at an attribute is used to estimate the size of the minimal tree under that attribute and an attribute with the smallest expected tree is selected we present two types of anytime induction algorithms a contract algorithm that determines the sample size on the basis of a pre given allocation of time and an interruptible algorithm that starts with a greedy tree and continuously improves subtrees by additional sampling experimental results indicate that for several hard concepts our proposed approach exhibits good anytime behavior and yields significantly better decision trees when more time is available

ifrowann imbalanced fuzzy rough ordered weighted average nearest neighbor classification imbalanced classification deals with learning from data with a disproportional number of samples in its classes traditional classifiers exhibit poor behavior when facing this kind of data because they do not take into account the imbalanced class distribution four main kinds of solutions exist to solve this problem modifying the data distribution modifying the learning algorithm for considering the imbalance representation including the use of costs for data samples and ensemble methods in this paper we adopt the second type of solution and introduce a classification algorithm for imbalanced data that uses fuzzy rough set theory and ordered weighted average aggregation the proposal considers different strategies to build a weight vector to take into account data imbalance our methods are validated by an extensive experimental study showing statistically better results than 13 other state of the art methods

labelling strategies for hierarchical multi label classification techniques many hierarchical multi label classification systems predict a real valued score for every instance class couple with a higher score reflecting more confidence that the instance belongs to that class these classifiers leave the conversion of these scores to an actual label set to the user who applies a cut off value to the scores the predictive performance of these classifiers is usually evaluated using threshold independent measures like precision recall curves however several applications require actual label sets and thus an automatic labelling strategy in this paper we present and evaluate different alternatives to perform the actual labelling in hierarchical multi label classification we investigate the selection of both single and multiple thresholds despite the existence of multiple threshold selection strategies in non hierarchical multi label classification they cannot be applied directly to the hierarchical context the proposed strategies are implemented within two main approaches optimisation of a certain performance measure of interest such as f measure or hierarchical loss and simulating training set properties such as class distribution or label cardinality in the predictions we assess the performance of the proposed labelling schemes on 10 datasets from different application domains our results show that selecting multiple thresholds may result in an efficient and effective solution for hierarchical multi label problems highlightswe select single or multiple thresholds for hierarchical multi label classifiers selecting multiple thresholds often yield better label sets in lesser time we show that optimising h loss tends to favor empty label sets multiple threshold selection is preferred for micro f measure and hmc loss imitating training set properties is a competitive approach to optimise hmc loss

a novel approach to statistical comparison of meta heuristic stochastic optimization algorithms using deep statistics abstract in this paper a novel approach for making a statistical comparison of meta heuristic stochastic optimization algorithms over multiple single objective problems is introduced where a new ranking scheme is proposed to obtain data for multiple problems the main contribution of this approach is that the ranking scheme is based on the whole distribution instead of using only one statistic to describe the distribution such as average or median averages are sensitive to outliers i e the poor runs of the stochastic optimization algorithms and consequently medians are sometimes used however using the common approach with either averages or medians the results can be affected by the ranking scheme that is used by some standard statistical tests this happens when the differences between the averages or medians are in some neighborhood and the algorithms obtain different ranks though they should be ranked equally given the small differences that exist between them the experimental results obtained on black box benchmarking 2015 show that our approach gives more robust results compared to the common approach in cases when the results are affected by outliers or by a misleading ranking scheme

a fuzzy genetic programming based algorithm for subgroup discovery and the application to one problem of pathogenesis of acute sore throat conditions in humans this paper proposes a novel algorithm for subgroup discovery task based on genetic programming and fuzzy logic called fuzzy genetic programming based for subgroup discovery fugepsd the genetic programming allows to learn compact expressions with the main objective to obtain rules for describing simple interesting and interpretable subgroups this algorithm incorporates specific operators in the search process to promote the diversity between the individuals the evolutionary scheme of fugepsd is codified through the genetic cooperative competitive approach promoting the competition and cooperation between the individuals of the population in order to find out the optimal solutions for the sd task fugepsd displays its potential with high quality results in a wide experimental study performed with respect to others evolutionary algorithms for subgroup discovery moreover the quality of this proposal is applied to a case study related to acute sore throat problems

multi target support vector regression via correlation regressor chains abstract multi target regression is a challenging task that consists of creating predictive models for problems with multiple continuous target outputs despite the increasing attention on multi label classification there are fewer studies concerning multi target mt regression the current leading mt models are based on ensembles of regressor chains where random differently ordered chains of the target variables are created and used to build separate regression models using the previous target predictions in the chain the challenges of building mt models stem from trying to capture and exploit possible correlations among the target variables during training this paper presents three multi target support vector regression models the first involves building independent single target support vector regression svr models for each output variable the second builds an ensemble of random chains using the first method as a base model the third calculates the targets correlations and forms a maximum correlation chain which is used to build a single chained support vector regression model improving the models prediction performance while reducing the computational complexity the experimental study evaluates and compares the performance of the three approaches with seven other state of the art multi target regressors on 24 multi target datasets the experimental results are then analyzed using non parametric statistical tests the results show that the maximum correlation svr approach improves the performance of using ensembles of random chains