kernel based hard clustering methods in the feature space with automatic variable weighting abstract this paper presents variable wise kernel hard clustering algorithms in the feature space in which dissimilarity measures are obtained as sums of squared distances between patterns and centroids computed individually for each variable by means of kernels the methods proposed in this paper are supported by the fact that a kernel function can be written as a sum of kernel functions evaluated on each variable separately the main advantage of this approach is that it allows the use of adaptive distances which are suitable to learn the weights of the variables on each cluster providing a better performance moreover various partition and cluster interpretation tools are introduced experiments with synthetic and benchmark datasets show the usefulness of the proposed algorithms and the merit of the partition and cluster interpretation tools

fuzzy clustering in a complex network based on content relevance and link structures many real world problems can be represented as complex networks with nodes representing different objects and links between nodes representing relationships between objects as different attributes can be considered as associating with different objects other than nontrivial link structures complex networks also contain rich content information and it can be a big challenge to find interesting clusters in such networks by fully exploiting the knowledge of both content and link information in them although some attempts have been made to tackle this clustering problem few of them have considered the feasibility of identifying clusters in complex networks using a fuzzy based clustering approach we believe that if the degree of membership to a cluster that a node belongs to can be considered we will be able to better identify clusters in complex networks as we may be able to identify overlapping clusters in this paper we therefore propose a fuzzy based clustering algorithm for this task the algorithm which we call fuzzy clustering algorithm for complex networks fcan can discover clusters by taking into consideration both link and content information it does so by first processing the content information by introducing a measure to quantify the relevance of contents between each pair of nodes within the network it then proceeds to leverage the link information in the clustering process by considering a measure of cluster density based on these measures fcan identifies fuzzy clusters that are more densely connected and more highly relevant in their contents to optimize the degrees of memberships of each node belonging to different clusters the performance of fcan has been evaluated with several synthetic and real datasets involving those of document classification and social community detection the results show that in terms of accuracy computation efficiency and scalability fcan can be a very promising approach

a clustering fuzzification algorithm based on alm abstract in this paper we propose a fuzzification method for clusters produced from a clustering process based on active learning method alm alm is a soft computing methodology which is based on a hypothesis claiming that human brain interprets information in pattern like images the proposed fuzzification method is applicable to all non fuzzy clustering algorithms as a post process the most outstanding advantage of this method is the ability to determine the membership degrees of each data to all clusters based on the density and shape of the clusters it is worth mentioning that for existing fuzzy clustering algorithms such as fcm the membership degree is usually determined as a function of distance to the center of the clusters in our proposed method all data points of a cluster will play a role in order to determine the membership degrees consequently the obtained membership degrees will depend on all of the data points of clusters the amount of data points and the density distribution of the clusters simulations prove the advantages of the proposed method

constructing a fuzzy controller from data abstract fuzzy control at the executive level can be interpreted as an approximation technique for a control function based on typical imprecisely specified input output tuples that are represented by fuzzy sets the imprecision is characterized by similarity relations that are induced by transformations of the canonical distance function between real numbers taking this interpretation of fuzzy controllers into account in order to derive a fuzzy controller from observed data typical input output tuples have to be identified in addition a concept of similarity based on a transformations of the canonical distance is needed in order to characterize the typical input output tuples by suitable fuzzy sets a variety of fuzzy clustering algorithms that are exactly working in this spirit exists they identify prototypes and assign fuzzy sets to the prototypes on the basis of a suitable transformed distance in this paper we discuss how such fuzzy clustering techniques can be applied to construct a fuzzy controller from data and introduce special clustering algorithms that are tailored for this problem

on a class of optimal nonbinary linear unequal error protection codes for two sets of messages several authors have addressed the problem of designing good linear unequal error protection luep codes however very little is known about good nonbinary luep codes the authors present a class of optimal nonbinary luep codes for two different sets of messages by combining t error correcting reed solomon rs codes and shortened nonbinary hamming codes they obtain nonbinary luep codes that protect one set of messages against any t or fewer symbol errors and the remaining set of messages against any single symbol error for t spl ges 2 they show that these codes are optimal in the sense of achieving the hamming lower bound on the number of redundant symbols of a nonbinary luep code with the same parameters

improved upper bounds for codes with unequal error protection asymptotic nonexistence bounds for unequal error protecting codes with two protection levels are considered we show that the improved estimates on the possible distance distributions for codes may sometimes yield sharper upper bounds than the previously known ones on the higher significance protection level of both nonlinear and linear codes having two protection levels

order selection in finite mixture models with a nonsmooth penalty order selection is a fundamental and challenging problem in the application of finite mixture models in this article we develop a new penalized likelihood approach the new method modified smoothly clipped absolute deviation mscad deviates from information based methods such as akaike information criterion aic and bayesian information criterion bic by introducing two penalty functions that depend on the mixing proportions and the component parameters it is consistent at estimating both the order of the mixture model and the mixing distribution simulations show that mscad has much better performance than a number of existing methods two real data examples are examined to illustrate the performance of mscad

testing the order of a model using locally conic parametrization population mixtures and stationary arma processes in this paper we address the problem of testing hypotheses using the likelihood ratio test statistic in nonidentifiable models with application to model selection in situations where the parametrization for the larger model leads to nonidentifiability in the smaller model we give two major applications the case where the number of populations has to be tested in a mixture and the case of stationary arma p q processes where the order p q has to be tested we give the asymptotic distribution for the likelihood ratio test statistic when testing the order of the model in the case of order selection for armas the asymptotic distribution is invariant with respect to the parameters generating the process a locally conic parametrization is a key tool in deriving the limiting distributions it allows one to discover the deep similarity between the two problems

new upper bounds on error exponents we derive new upper bounds on the error exponents for the maximum likelihood decoding and error detecting in the binary symmetric channels this is an improvement on the best earlier known bounds by shannon gallager berlekamp 1967 and mceliece omura 1977 for the probability of undetected error the new bounds are better than the bounds by levenshtein 1978 1989 and the bound by abdel ghaffar see ibid vol 43 p 1489 502 1997 moreover we further extend the range of rates where the undetected error exponent is known to be exact the new bounds are based on an analysis of possible distance distributions of the codes along with some inequalities relating the distance distributions to the error probabilities

coding for the binary symmetric broadcast channel with two receivers block coding for the binary symmetric broadcast channel with two receivers is investigated a graph theoretic approach to the construction of a class of block codes with unequal error protection for two different sets of messages is presented a code in this class is a direct sum of two component codes each set of messages is encoded based on one component code the codes in this class are easy to implement decoding of these codes is presented and lower bounds on the achievable rates of these codes are derived the bounds are tighter than the katsman s bounds

kernel fuzzy c means with automatic variable weighting this paper presents variable wise kernel fuzzy c means clustering methods in which dissimilarity measures are obtained as sums of euclidean distances between patterns and centroids computed individually for each variable by means of kernel functions the advantage of the proposed approach over the conventional kernel clustering methods is that it allows us to use adaptive distances which change at each algorithm iteration and can either be the same for all clusters or different from one cluster to another this kind of dissimilarity measure is suitable to learn the weights of the variables during the clustering process improving the performance of the algorithms another advantage of this approach is that it allows the introduction of various fuzzy partition and cluster interpretation tools experiments with synthetic and benchmark datasets show the usefulness of the proposed algorithms and the merit of the fuzzy partition and cluster interpretation tools

hypothesis test for normal mixture models the em approach normal mixture distributions are arguably the most important mixture models and also the most technically challenging the likelihood function of the normal mixture model is unbounded based on a set of random samples unless an artificial bound is placed on its component variance parameter moreover the model is not strongly identifiable so it is hard to differentiate between over dispersion caused by the presence of a mixture and that caused by a large variance and it has infinite fisher information with respect to mixing proportions there has been extensive research on finite normal mixture models but much of it addresses merely consistency of the point estimation or useful practical procedures and many results require undesirable restrictions on the parameter space we show that an em test for homogeneity is effective at overcoming many challenges in the context of finite normal mixtures we find that the limiting distribution of the em test is a simple function of the 0 5 2 0 5 2 and 2 distributions when the mixing variances are equal but unknown and the 2 when variances are unequal and unknown simulations show that the limiting distributions approximate the finite sample distribution satisfactorily two genetic examples are used to illustrate the application of the em test

signatures definitions operators and applications to fuzzy modelling this paper presents a new framework for the symbolic representation of data which is referred to as signatures the definitions of signatures and of signature trees are first given original operators on signatures are next presented i e contraction extension pruning addition multiplication and grafting attractive applications of signatures related to the modelling of fuzzy inference systems are suggested and discussed an example is included to accompany the theoretical results

a constrained formulation of maximum likelihood estimation for normal mixture distributions in this context the normal densities are sometimes referred to as component densities the log likelihood function l zy corresponding to a random sample xl xnj is defined by l ty ek 1 10g pm xk y the first problem associated with maximum likelihood estimation arises from the unboundedness of l qy on r day 1969 a global maximum likelihood estimate always fails to exist in addition the unboundedness of l y causes failures of optimization algorithms of both the em redner and walker 1984 and quasi newton fowlkes 1979 types in spite of the unboundedness of l 4y statistical theory kiefer 1978 guarantees that a particular local maximizer of l y is strongly consistent and asymptotically efficient several local maximizers can exist for a given sample and the other major maximum likelihood difficulty is in determining when the correct one has been found day 1969 noted that spurious maximizers corresponding to parameter points having some component standard deviations very small relative to others are generated by any small number of sample points grouped sufficiently close together the spurious maximizers like the unboundedness of l zy can create difflculties when using the em or quasi newton algorithms the constrained maximum likelihood formulation presented in the next section avoids the unconstrained problems of singularities and spurious maximizers of the type noted by day section 3 contains the statement and proof of a consistency result for the corresponding constrained estimator

mercer kernel based clustering in feature space the article presents a method for both the unsupervised partitioning of a sample of data and the estimation of the possible number of inherent clusters which generate the data this work exploits the notion that performing a nonlinear data transformation into some high dimensional feature space increases the probability of the linear separability of the patterns within the transformed space and therefore simplifies the associated data structure it is shown that the eigenvectors of a kernel matrix which defines the implicit mapping provides a means to estimate the number of clusters inherent within the data and a computationally simple iterative procedure is presented for the subsequent feature space partitioning of the data

fitting the erlang mixture model to data via a gem cmm algorithm abstract the erlang mixture model with common scale parameter is flexible and analytically tractable as such it is a useful model to fit insurance loss data and to calculate quantities of interest for insurance risk management in this paper we propose a generalized expectation maximization gem algorithm along with a clusterized method of moments cmm to estimate the model parameters the gem algorithm not only estimates the mixing weights and scale parameter of the model but also estimates the shape parameters of the model using a local search method the cmm method enables to produce quality initial estimates for the gem algorithm as a result the proposed approach provides an efficient algorithm that can fit the model to the body and the tail of truncated and censored loss data well and converges fast we examine the performance of the proposed approach through several simulation studies and apply it to fit the erlang mixture model to two real loss data sets

clustering via hilbert space abstract we discuss novel clustering methods that are based on mapping data points to a hilbert space by means of a gaussian kernel the first method support vector clustering svc searches for the smallest sphere enclosing data images in hilbert space the second quantum clustering qc searches for the minima of a potential function defined in such a hilbert space in svc the minimal sphere when mapped back to data space separates into several components each enclosing a separate cluster of points a soft margin constant helps in coping with outliers and overlapping clusters in qc minima of the potential define cluster centers and equipotential surfaces are used to construct the clusters in both methods the width of the gaussian kernel controls the scale at which the data are probed for cluster formations we demonstrate the performance of the algorithms on several data sets

testing the order of a model this paper deals with order identification for nested models in the i i d framework we study the asymptotic efficiency of two generalized likelihood ratio tests of the order they are based on two estimators which are proved to be strongly consistent a version of stein s lemma yields an optimal underestimation error exponent the lemma also implies that the overestimation error exponent is necessarily trivial our tests admit non trivial underestimation error exponents the optimal underestimation error exponent is achieved in some situations the overestimation error can decay exponentially fast with respect to a positive power of the number of observations these results are proved under mild assumptions by relating the underestimation resp overestimation error to large resp moderate deviations of the log likelihood process in particular it is not necessary that the classical cramer condition be satisfied namely the log densities are not required to admit every exponential moment three benchmark examples with specific difficulties location mixture of normal distributions abrupt changes and various regressions are detailed so as to illustrate the generality of our results

inference for subgroup analysis with a structured logistic normal mixture model in this article we propose a statistical model for the purpose of identifying a subgroup that has an enhanced treatment effect as well as the variables that are predictive of the subgroup membership the need for such subgroup identification arises in clinical trials and in market segmentation analysis by using a structured logistic normal mixture model our proposed framework enables us to perform a confirmatory statistical test for the existence of subgroups and at the same time to construct predictive scores for the subgroup membership the inferential procedure proposed in the article is built on the recent literature on hypothesis testing for gaussian mixtures but the structured logistic normal mixture model enjoys some distinctive properties that are unavailable to the simpler gaussian mixture models with the bootstrap approximations the proposed tests are shown to be powerful and equally importantly insensitive to the choice of tuning parameters as an illustration we analyze a dataset fr

testing for a finite mixture model with two components summary we consider a finite mixture model with k components and a kernel distribution from a general one parameter family the problem of testing the hypothesis k 2 versus k 3 is studied there has been no general statistical testing procedure for this problem we propose a modified likelihood ratio statistic where under the null and the alternative hypotheses the estimates of the parameters are obtained from a modified likelihood function it is shown that estimators of the support points are consistent the asymptotic null distribution of the modified likelihood ratio test proposed is derived and found to be relatively simple and easily applied simulation studies for the asymptotic modified likelihood ratio test based on finite mixture models with normal binomial and poisson kernels suggest that the test proposed performs well simulation studies are also conducted for a bootstrap method with normal kernels an example involving foetal movement data from a medical study illustrates the testing procedure

enhancement of the classification and reconstruction performance of fuzzy c means with refinements of prototypes abstract owning to their abilities to reveal structural relationships in data fuzzy clustering plays a pivotal role in fuzzy modeling pattern recognition and data analysis as supporting an unsupervised mode of learning fuzzy clustering brings about unique opportunities to build a structural backbone of numerous constructs in the areas identified above a follow up phase is required when the structural findings developed in the form of fuzzy clusters need some refinements usually when the clustering results are used afterwards in the model being developed in a supervised mode following this general line of thought in this study we propose a novel approach to optimize the clustering and classification performance of the fuzzy c means fcm algorithm proceeding with a collection of clusters information granules produced by the fcm we carry out the refinements of the results in order to improve the representation or classification capabilities of fuzzy clusters by adjusting a location of the prototypes so that a certain performance index becomes optimized at this phase the optimization is carried out in a supervised mode with the aid of differential evolution de we propose five different strategies to adjust a location of the prototypes experimental studies completed on synthetic data and publicly available real world data quantify the improvement of the representation and classification abilities of the clustering method

a survey of kernel and spectral methods for clustering clustering algorithms are a useful tool to explore data structures and have been employed in many disciplines the focus of this paper is the partitioning clustering problem with a special interest in two recent approaches kernel and spectral methods the aim of this paper is to present a survey of kernel and spectral clustering methods two approaches able to produce nonlinear separating hypersurfaces between clusters the presented kernel clustering methods are the kernel version of many classical clustering algorithms e g k means som and neural gas spectral clustering arise from concepts in spectral graph theory and the clustering problem is configured as a graph cut problem where an appropriate objective function has to be optimized an explicit proof of the fact that these two paradigms have the same objective is reported since it has been proven that these two seemingly different approaches have the same mathematical foundation besides fuzzy kernel clustering methods are presented as extensions of kernel k means clustering algorithm

support vector clustering we present a novel clustering method using the approach of support vector machines data points are mapped by means of a gaussian kernel to a high dimensional feature space where we search for the minimal enclosing sphere this sphere when mapped back to data space can separate into several components each enclosing a separate cluster of points we present a simple algorithm for identifying these clusters the width of the gaussian kernel controls the scale at which the data is probed while the soft margin constant helps coping with outliers and overlapping clusters the structure of a dataset is explored by varying the two parameters maintaining a minimal number of support vectors to assure smooth cluster boundaries we demonstrate the performance of our algorithm on several datasets

multiple kernel fuzzy clustering while fuzzy c means is a popular soft clustering method its effectiveness is largely limited to spherical clusters by applying kernel tricks the kernel fuzzy c means algorithm attempts to address this problem by mapping data with nonlinear relationships to appropriate feature spaces kernel combination or selection is crucial for effective kernel clustering unfortunately for most applications it is uneasy to find the right combination we propose a multiple kernel fuzzy c means mkfc algorithm that extends the fuzzy c means algorithm with a multiple kernel learning setting by incorporating multiple kernels and automatically adjusting the kernel weights mkfc is more immune to ineffective kernels and irrelevant features this makes the choice of kernels less crucial in addition we show multiple kernel k means to be a special case of mkfc experiments on both synthetic and real world data demonstrate the effectiveness of the proposed mkfc algorithm

fuzzy rule based models with interactive rules and their granular generalization abstract processing realized in fuzzy rule based models is associated with matching individual rules with the existing input data and aggregation of the levels of matching and conclusions of the rules rules are processed as individual entities in this study we introduce an augmentation of fuzzy models by facilitating interaction among the rules leading to more flexible type membership functions of fuzzy sets forming conditions of the rules thus resulting in substantially advanced topology of the partition of the input space different ways of realizing interaction among the rules are studied in the sequel we develop a granular fuzzy model implied by the rule based model showing how granular parameters of the original rule based model enhance its quality expressed in terms of coverage of experimental data the two evaluation criteria of the constructed granular model namely coverage and specificity are studied experimental results are reported for a series of publicly available data

fuzziness in data analysis the first aim is to emphasize the use of fuzziness in data analysis to capture information that has been traditionally disregarded with a cost in the precision of the conclusions fuzziness can be considered in the data analysis process at various stages but the main target in this paper will be fuzziness in the data depending on the nature of the fuzzy data or the aim to which they are handled different approaches should be applied we attempt to contribute to the clarification of such a difference while focusing on the so called ontic approach in contrast to the epistemic approach the second aim is to underline the need of considering robust methods to reduce the misleading impact of outliers in fuzzy data analysis we propose trimming as a general and intuitive method to discard outliers we exemplify this approach with the case of the ontic fuzzy trimmed mean variance and highlight the differences with the epistemic case all the discussions and developments are illustrated by means of a case study concerning the perception of lengths of men and women

bounds on distance distributions in codes of known size we treat the problem of bounding components of the possible distance distributions of codes given the knowledge of their size and possibly minimum distance using the beckner inequality from harmonic analysis we derive upper bounds on distance distribution components which are sometimes better than earlier ones due to ashikhmin barg and litsyn we use an alternative approach to derive upper bounds on distance distributions in linear codes as an application of the suggested estimates we get an upper bound on the undetected error probability for an arbitrary code of given size we also use the new bounds to derive better upper estimates on the covering radius as well as a lower bound on the error probability threshold as a function of the code s size and minimum distance

testing the order of a finite mixture the order is an important parameter in applications of finite mixture models yet designing a valid and easy to use statistical test for the order is challenging to date most results on hypothesis tests have focused on homogeneity a special case where the null model has order 1 in this work we designed an em test for the general problem of testing the null hypothesis of order m0 versus an alternative hypothesis of order larger than m0 for any positive integer m0 the null limiting distribution of the em test is a mixture of 2 distributions the weights in this mixture limiting distribution can be conveniently computed compared with related results the new result is obtained under much less strict requirements on the component distribution and the parameter space extensive simulation studies show that the limiting distributions closely match the finite sample distributions of the em test when m0 2 the new em test has more accurate type i errors and matches the power of the modified likelihood

nonlinear component analysis as a kernel eigenvalue problem a new method for performing a nonlinear form of principal component analysis is proposed by the use of integral operator kernel functions one can efficiently compute principal components in high dimensional feature spaces related to input space by some nonlinear map for instance the space of all possible five pixel products in 16 16 images we give the derivation of the method and present experimental results on polynomial feature extraction for pattern recognition

enhancement of fuzzy clustering by mechanisms of partial supervision semi supervised or partial fuzzy clustering plays an important and unique role in discovering hidden structure in data realized in presence of a certain quite limited fraction of labeled patterns the objective of this study is to investigate and quantify the effect of various distance functions distances on the performance of the clustering mechanisms the underlying goal of endowing the clustering algorithms with a higher level of flexibility is done via the use of various distances the enhancement of this character is evaluated by means of a comprehensive assessment of quality of clusters their ensuing discrimination abilities and the accuracy of clusters themselves in addition to the standard euclidean distance being commonly exploited in fuzzy clustering three more versatile and adaptive distance measures are considered such as its weighted version a full adaptive distance and a kernel based distance using fuzzy c means fcm coming in its generic format we show its semi supervised enhancements derive detailed formulas and analyze their effectiveness the improvements of semi supervised clustering are empirically evaluated and numerically quantified with the use of several machine learning data sets

empirical bayes analysis of a microarray experiment microarrays are a novel technology that facilitates the simultaneous measurement of thousands of gene expression levels a typical microarray experiment can produce millions of data points raising serious problems of data reduction and simultaneous inference we consider one such experiment in which oligonucleotide arrays were employed to assess the genetic effects of ionizing radiation on seven thousand human genes a simple nonparametric empirical bayes model is introduced which is used to guide the efficient reduction of the data to a single summary statistic per gene and also to make simultaneous inferences concerning which genes were affected by the radiation although our focus is on one specific experiment the proposed methods can be applied quite generally the empirical bayes inferences are closely related to the frequentist false discovery rate fdr criterion

asymptotics for likelihood ratio tests under loss of identifiability this paper describes the large sample properties of the likelihood ratio test statistic lrts when the parameters characterizing the true null distribution are not unique it is well known that the classical asymptotic theory for the likelihood ratio test does not apply to such problems and the lrts may not have the typical chi squared type limiting distribution this paper establishes a general quadratic approximation of the log likelihood ratio function in a hellinger neighborhood of the true density which is valid with or without loss of identifiability of the true distribution under suitable conditions the asymptotic null distribution of the lrts under loss of identifiability can be obtained by maximizing the quadratic form these results extend the work of chernoff and le cam in particular applications to testing the number of mixture components in finite mixture models are discussed

rapid and brief communication evaluation of the performance of clustering algorithms in kernel induced feature space by using a kernel function data that are not easily separable in the original space can be clustered into homogeneous groups in the implicitly transformed high dimensional feature space kernel k means algorithms have recently been shown to perform better than conventional k means algorithms in unsupervised classification however few reports have examined the benefits of using a kernel function and the relative merits of the various kernel clustering algorithms with regard to the data distribution in this study we reformulated four representative clustering algorithms based on a kernel function and evaluated their performances for various data sets the results indicate that each kernel clustering algorithm gives markedly better performance than its conventional counterpart for almost all data sets of the kernel clustering algorithms studied in the present work the kernel average linkage algorithm gives the most accurate clustering results

robust estimation of mixture complexity in many applications it is important to find the mixture with fewest number of components known as the mixture complexity that provides a satisfactory fit to the data this article focuses on developing an estimator of mixture complexity that is consistent when the form of component densities are unknown but are postulated to be members of some parametric family and is simultaneously robust against model misspecification we treat the estimation of mixture complexity as a model selection problem and construct an estimator of mixture complexity as a byproduct of minimizing a hellinger information criterion this estimator is shown to be consistent for any parametric family of mixtures when the model is correctly specified monte carlo simulations for a wide variety of normal mixtures show that our estimator is very competitive with several others in the literature in correctly identifying the true mixture complexity the basic construction being firmly rooted in the minimum hellinger distance approach

constructing linear unequal error protection codes from algebraic curves we show that the concept of generalized algebraic geometry codes which was introduced by xing niederreiter and lam see ibid vol 45 p 2498 2501 nov 1999 gives a natural framework for constructing linear unequal error protection codes

kernel based fuzzy clustering and fuzzy clustering a comparative experimental study in this study we present a comprehensive comparative analysis of kernel based fuzzy clustering and fuzzy clustering kernel based clustering has emerged as an interesting and quite visible alternative in fuzzy clustering however the effectiveness of this extension vis a vis some generic methods of fuzzy clustering has neither been discussed in a complete manner nor the performance of clustering quantified through a convincing comparative analysis our focal objective is to understand the performance gains and the importance of parameter selection for kernelized fuzzy clustering generic fuzzy c means fcm and gustafson kessel gk fcm are compared with two typical generalizations of kernel based fuzzy clustering one with prototypes located in the feature space kfcm f and the other where the prototypes are distributed in the kernel space kfcm k both generalizations are studied when dealing with the gaussian kernel while kfcm k is also studied with the polynomial kernel two criteria are used in evaluating the performance of the clustering method and the resulting clusters namely classification rate and reconstruction error through carefully selected experiments involving synthetic and machine learning repository http archive ics uci edu beta data sets we demonstrate that the kernel based fcm algorithms produce a marginal improvement over standard fcm and gk for most of the analyzed data sets it has been observed that the kernel based fcm algorithms are in a number of cases highly sensitive to the selection of specific values of the kernel parameters

a note on the gustafson kessel and adaptive fuzzy clustering algorithms in this letter we show that the gustafson kessel g k algorithm 1979 and the original adaptive fuzzy clustering afc algorithm can be thought of as special cases of a more general algorithm our analysis shows that the g k algorithm is better suited for ellipsoidal clusters of equal volume whereas the original afc algorithm is better suited for linear clusters we also discuss a new variation of these algorithms which can be used to improve the results of the g k and afc algorithms in some cases

nonlinear unequal error protection codes are sometimes better than linear ones the parameters of some binary nonlinear two level unequal error protection uep codes are evaluated and compared with nonexistence bounds for linear uep codes the major points is that for uep codes linearity is an essential restriction the results demonstrate that at least in certain cases nonlinear codes perform better than linear ones this is in contrast to the situation for ordinary codes when in most cases the best possible result can be obtained even with the restriction of linearity

non finite fisher information and homogeneity an em approach summary even simple examples of finite mixture models can fail to fulfil the regularity conditions that are routinely assumed in standard parametric inference problems many methods have been investigated for testing for homogeneity in finite mixture models for example but all rely on regularity conditions including the finiteness of the fisher information and the space of the mixing parameter being a compact subset of some euclidean space very simple examples where such assumptions fail include mixtures of two geometric distributions and two exponential distributions and more generally mixture models in scale distribution families to overcome these difficulties we propose and study an em test statistic which has a simple limiting distribution for examples in this paper simulations show that the em test has accurate type i errors and is more efficient than existing methods when they are applicable a real example is also included

testing for observation dependent regime switching in mixture autoregressive models abstract testing for regime switching when the regime switching probabilities are specified either as constants mixture models or are governed by a finite state markov chain markov switching models are long standing problems that have also attracted recent interest this paper considers testing for regime switching when the regime switching probabilities are time varying and depend on observed data observation dependent regime switching specifically we consider the likelihood ratio test for observation dependent regime switching in mixture autoregressive models the testing problem is highly nonstandard involving unidentified nuisance parameters under the null parameters on the boundary singular information matrices and higher order approximations of the log likelihood we derive the asymptotic null distribution of the likelihood ratio test statistic in a general mixture autoregressive setting using high level conditions that allow for various forms of dependence of the regime switching probabilities on past observations and we illustrate the theory using two particular mixture autoregressive models the likelihood ratio test has a nonstandard asymptotic distribution that can easily be simulated and monte carlo studies show the test to have good finite sample size and power properties

residual diagnostics for mixture models abstract a sample is commonly modeled by a mixture distribution if the observations follow a common distribution but the parameter of interest differs between observations for example we observe the lengths but not the ages of a sample offish it may be reasonable to assume that length is normally distributed about an unknown mean that depends on the age of the fish provided there is more than one age class in the sample then the data are distributed as a mixture of normals in this article we assume that the data are a random sample from a mixture of exponential family distributions and that for each observation the parameter of interest is sampled independently from an unknown mixing distribution q the adequacy of a fitted mixture model can be assessed by examining residuals based on the ratio of the observed to expected fit residuals based on the homogeneity model in which q is a one point distribution display a convexity property when the data follow a mixture model this becomes the basis fo

structural multiple empirical kernel learning multiple kernel learning mkl can boost classification performance through using multiple kernels rather than a single fixed one unlike the traditional mkl with the implicit kernels multiple empirical kernel learning mekl explicitly maps input data into multiple feature spaces this paper focuses on mekl and proposes an effective threefold structural mekl tsmekl the first fold structure is the space structural information between different mapped feature spaces the second one is the class discriminant information within each mapped feature space the third one is the cluster structural information of samples in each mapped feature space the classical mekl mainly pays attention to the first two structures but neglects the last one the proposed tsmekl introduces the cluster structural information into mekl doing so can simultaneously utilize the space the class and the cluster information in the way from globality to locality therefore tsmekl utilizes threefold structural information to result in the improvement of classification performance to the best of our knowledge it is the first time to introduce the cluster information into the mekl framework the main advantage of the developed tsmekl is considering different folds of data information to improve classification performance the experimental results validate the feasibility and effectiveness of tsmekl moreover we discuss the theoretical and experimental generalization risk bound of the proposed algorithm in terms of the rademacher complexity

clustering with proximity knowledge and relational knowledge in this article a proximity fuzzy framework for clustering relational data is presented where the relationships between the entities of the data are given in terms of proximity values we offer a comprehensive and in depth comparison of our clustering framework with proximity relational knowledge to clustering with distance relational knowledge such as the well known relational fuzzy c means fcm we conclude that proximity can provide a richer description of the relationships among the data and this offers a significant advantage when realizing clustering we further motivate clustering relational proximity data and provide both synthetic and real world experiments to demonstrate both the usefulness and advantage offered by clustering proximity data finally a case study of relational clustering is introduced where we apply proximity fuzzy clustering to the problem of clustering a set of trees derived from software requirements engineering the relationships between trees are based on the degree of closeness in both the location of the nodes in the trees and the semantics associated with the type of connections between the nodes

distance metric learning for soft subspace clustering in composite kernel space soft subspace clustering algorithms have been successfully used for high dimensional data in recent years however the existing algorithms often utilize only one distance function to evaluate the distance between data items on each feature which cannot deal with datasets with complex inner structures in this paper a composite kernel space cks is constructed based on a set of basis kernels and a novel framework of soft subspace clustering is proposed by integrating distance metric learning in the cks two soft subspace clustering algorithms i e entropy weighting fuzzy clustering in cks for kernel space cks ewfc k and feature space cks ewfc f are thus developed in both algorithms the prototype in the feature space is mapped into the cks by multiple simultaneous mappings one mapping for each cluster which is distinct from existing kernel based clustering algorithms by evaluating the distance on each feature in the cks both cks ewfc k and cks ewfc f learn the distance function adaptively during the clustering process experimental results have demonstrated that the proposed algorithms in general outperform classical clustering algorithms and are immune to ineffective kernels and irrelevant features in soft subspace the composite kernel space is constructed based on a set of basis kernels the general form of soft subspace clustering in cks is presented cks ewfc k and cks ewfc f are proposed under the framework of cks ssc the properties of cks ewfc k and cks ewfc f are investigated both cks ewfc k and cks ewfc f are immune to ineffective kernels

constructive codes with unequal error protection an asymptotic lower bound on the rates of two level binary codes with unequal error protection is presented the bound is based on a class of codes that are constructive in the sense that the description complexity is polynomial in the codeword length in some cases the bound exceeds an upper bound for linear codes proving the existence of constructive nonlinear uep codes that are better than any linear codes

further improvements in feature weighted fuzzy c means in cluster analysis certain features of a given data set may exhibit higher relevance than others to address this issue feature weighted fuzzy c means fwfcm approaches have emerged in recent years however there are certain deficiencies in the existing fwfcms e g the elements in a feature weight vector cannot be adaptively adjusted during the training phase and the update formulas of a feature weight vector cannot be derived analytically in this study an improved fwfcm ifwfcm is proposed to overcome these shortcomings the ifwfcm kd based on the kernelized distance is also proposed experimental results reported for five numerical data sets and the color images show that ifwfcm is superior to the existing fwfcms an interesting conclusion that ifwfcm kd might not improve the performance of ifwfcm is also obtained by applying ifwfcm kd to tackle the above mentioned numerical data sets and color images

optimal rate of convergence for finite mixture models in finite mixture models we establish the best possible rate of convergence for estimating the mixing distribution we find that the key for estimating the mixing distribution is the knowledge of the number of components in the mixture while a sqrt n consistent rate is achievable when the exact number of components is known the best possible rate is only n 1 4 when it is unknown under a strong identifiability condition it is shown that this rate is reached by some minimum distance estimators most commonly used models are found to satisfy the strong identifiability condition

a novel fuzzy clustering algorithm with between cluster information for categorical data in this paper we present a new fuzzy clustering algorithm for categorical data in the algorithm the objective function of the fuzzy k modes algorithm is modified by adding the between cluster information so that we can simultaneously minimize the within cluster dispersion and enhance the between cluster separation for obtaining the local optimal solutions of the modified objective function the corresponding update formulas of the membership matrix and the cluster prototypes are strictly derived the convergence of the proposed algorithm under the optimization framework is proved on several real data sets from uci the performance of the proposed algorithm is studied the experimental results illustrate that the algorithm is effective and suitable for categorical data sets

testing the number of components in normal mixture regression models testing the number of components in finite normal mixture models is a long standing challenge because of its nonregularity this article studies likelihood based testing of the number of components in normal mixture regression models with heteroscedastic components we construct a likelihood based test of the null hypothesis of m0 components against the alternative hypothesis of m0 1 components for any m0 the null asymptotic distribution of the proposed modified em test statistic is the maximum of m0 random variables that can be easily simulated the simulations show that the proposed test has very good finite sample size and power properties supplementary materials for this article are available online

a novel kernel method for clustering kernel methods are algorithms that by replacing the inner product with an appropriate positive definite function implicitly perform a nonlinear mapping of the input data into a high dimensional feature space in this paper we present a kernel method for clustering inspired by the classical k means algorithm in which each cluster is iteratively refined using a one class support vector machine our method which can be easily implemented compares favorably with respect to popular clustering algorithms like k means neural gas and self organizing maps on a synthetic data set and three uci real data benchmarks iris data wisconsin breast cancer database spam database

optimizing the prototypes with a novel data weighting algorithm for enhancing the classification performance of fuzzy clustering abstract fuzzy clustering is regarded as an unsupervised learning process that constitutes a prerequisite for many other data mining techniques deciding how to classify data efficiently and accurately has been one of the topics pursued by many researchers we anticipate that the classification performance of the clustering is strongly dependent on the boundary data viz data located at the boundaries of the clusters the boundary data hold some levels of uncertainties and as such contain more information than others usually the greater the uncertainty the more information contained in such data to improve the quality of clustering this study develops an augmented scheme of fuzzy clustering in which a novel weighted data based fuzzy clustering is proposed in the introduced scheme a dataset is composed of boundary data and non boundary data the partition matrix is used to determine the boundary data and the non boundary data to be next considered in the clustering process then we assign different weights to each datum to construct the weighted data during this process we make the weights for the boundary data and the non boundary data different which makes the contributions of the boundary data and the non boundary data to the prototypes being reduced and enhanced respectively furthermore we build a weighting function to determine the weights of the data the weighted data are used to optimize the prototypes with the optimized prototypes the partition matrix can be refined which ultimately makes the boundaries of the clusters optimized finally the classification performance of fuzzy clustering is enhanced we offer a thorough analysis of the developed scheme comprehensive experimental studies involving synthetic and publicly available datasets are reported to demonstrate the performance of the proposed approach

active constrained fuzzy clustering in this paper we address the problem of constrained clustering along with active selection of clustering constraints in a unified framework to this aim we extend the improved possibilistic c means algorithm ipcm with multiple kernels learning setting under supervision of side information by incorporating multiple kernels the limitation of improved possibilistic c means to spherical clusters is addressed by mapping non linear separable data to appropriate feature space the proposed method is immune to inefficient kernels or irrelevant features by automatically adjusting the weight of kernels moreover extending ipcm to incorporate constraints its strong robustness and fast convergence properties are inherited by the proposed method in order to avoid querying inefficient or redundant clustering constraints an active query selection heuristic is embedded into the proposed method to query the most informative constraints experiments conducted on synthetic and real world datasets demonstrate the effectiveness of the proposed method highlightsactive constrained clustering is examined in this paper the proposed method relies on a multiple kernels learning setting the method deals with linear inseparable partially overlapping and noisy datasets an active query selection heuristic was embedded into the clustering algorithm the query selection heuristic is based on the measurement of mistake in clustering

an introduction to kernel based learning algorithms this paper provides an introduction to support vector machines kernel fisher discriminant analysis and kernel principal component analysis as examples for successful kernel based learning methods we first give a short background about vapnik chervonenkis theory and kernel feature spaces and then proceed to kernel based learning in supervised and unsupervised scenarios including practical and algorithmic considerations we illustrate the usefulness of kernel algorithms by discussing applications such as optical character recognition and dna analysis

a new kernel based fuzzy clustering approach support vector clustering with cell growing in this paper the support vector clustering is extended to an adaptive cell growing model which maps data points to a high dimensional feature space through a desired kernel function this generalized model is called multiple spheres support vector clustering which essentially identifies dense regions in the original space by finding their corresponding spheres with minimal radius in the feature space a multisphere clustering algorithm based on adaptive cluster cell growing method is developed whereby it is possible to obtain the grade of memberships as well as cluster prototypes in partition the effectiveness of the proposed algorithm is demonstrated for the problem of arbitrary cluster shapes and for prototype identification in an actual application to a handwritten digit data set

inference on the order of a normal mixture finite normal mixture models are used in a wide range of applications hypothesis testing on the order of the normal mixture is an important yet unsolved problem existing procedures often lack a rigorous theoretical foundation many are also hard to implement numerically in this article we develop a new method to fill the void in this important area an effective expectation maximization em test is invented for testing the null hypothesis of arbitrary order m 0 under a finite normal mixture model for any positive integer m 0 2 the limiting distribution of the proposed test statistic is we also use a novel computer experiment to provide empirical formulas for the tuning parameter selection the finite sample performance of the test is examined through simulation studies real data examples are provided the procedure has been implemented in r code the p values for testing the null order of m 0 2 or m 0 3 can be calculated with a single command this article has supplementary materials avail

the k means type algorithms versus imbalanced data distributions k means is a partitional clustering technique that is well known and widely used for its low computational cost the representative algorithms include the hard k means and the fuzzy k means however the performance of these algorithms tends to be affected by skewed data distributions i e imbalanced data they often produce clusters of relatively uniform sizes even if input data have varied cluster sizes which is called the uniform effect in this paper we analyze the causes of this effect and illustrate that it probably occurs more in the fuzzy k means clustering process than the hard k means clustering process as the fuzzy index m increases the uniform effect becomes evident to prevent the effect of the uniform effect we propose a multicenter clustering algorithm in which multicenters are used to represent each cluster instead of one single center the proposed algorithm consists of the three subalgorithms the fast global fuzzy k means best m plot and grouping multicenter algorithms they will be respectively used to address the three important problems 1 how are the reliable cluster centers from a dataset obtained 2 how are the number of clusters which these obtained cluster centers represent determined 3 how is it judged as to which cluster centers represent the same clusters the experimental studies on both synthetic and real datasets illustrate the effectiveness of the proposed clustering algorithm in clustering balanced and imbalanced data

fuzzy clustering of time series in the frequency domain traditional and fuzzy cluster analyses are applicable to variables whose values are uncorrelated hence in order to cluster time series data which are usually serially correlated one needs to extract features from the time series the values of which are uncorrelated the periodogram which is an estimator of the spectral density function of a time series is a feature that can be used in the cluster analysis of time series because its ordinates are uncorrelated additionally the normalized periodogram and the logarithm of the normalized periodogram are also features that can be used in this paper we consider a fuzzy clustering approach for time series based on the estimated cepstrum the cepstrum is the spectrum of the logarithm of the spectral density function we show in our simulation studies for the typical generating processes that have been considered fuzzy clustering based on the cepstral coefficients performs very well compared to when it is based on other features

kernel based hard clustering methods with kernelization of the metric and automatic weighting of the variables this paper presents kernel based hard clustering methods with kernelization of the metric and automatic weighting of the variables the proposed methodology is supported by the fact that a kernel function can be written as a sum of kernels evaluated separately on each variable thus in the proposed algorithms dissimilarity measures are obtained as sums of euclidean distances between patterns and centroids computed individually for each variable by means of kernels the main advantage of this approach over the conventional approach is that it allows the use of kernelized adaptive distances which are suitable to learn the weights of the variables dynamically improving the performance of the algorithms moreover various partition and cluster interpretation tools are introduced experiments with a number of benchmark datasets corroborate the usefulness of the proposed algorithms and the merit of the partition and cluster interpretation tools highlightspresents kernel based clustering methods with automatic weighting of the variables kernelized local and global adaptive distances are introduced the proposed algorithms are suitable to learn the weights of the variables partition and cluster interpretation tools are given experiments with several benchmark data sets corroborate the proposed methods

on unequal error protection of convolutional codes from an algebraic perspective in this paper convolutional codes are studied for unequal error protection uep from an algebraic theoretical viewpoint we first show that for every convolutional code there exists at least one optimal generator matrix with respect to uep the uep optimality of convolutional encoders is then combined with several algebraic properties e g systematic basic canonical and minimal to establish the fundamentals of convolutional codes for uep in addition a generic lower bound on the length of a uep convolutional code is proposed good uep codes with their lengths equal to the derived lower bound are obtained by computer search

consistent estimation of mixture complexity the consistent estimation of mixture complexity is of fundamental importance in many applications of finite mixture models an enormous body of literature exists regarding the application computational issues and theoretical aspects of mixture models when the number of components is known but estimating the unknown number of components remains an area of intense research effort this article presents a semiparametric methodology yielding almost sure convergence of the estimated number of components to the true but unknown number of components the scope of application is vast as mixture models are routinely employed across the entire diverse application range of statistics including nearly all of the social and experimental sciences

conjecturable knowledge discovery a fuzzy clustering approach traditionally clustering is the task of dividing objects into homogeneous clusters based on their degrees of similarity as objects are assigned to clusters users need to manually give descriptions for all clusters characterizing clusters by hand can consume a great deal of time of users in addition users sometimes have no specific idea as to how to explain the clustering results thus they might give inappropriate descriptions a clustering technique is proposed to discover conjecturable rules providing descriptions of clusters with a decision tree classification technique every cluster in a conjecturable tree is depicted by only one conjecturable rule however less utilized rules are not necessarily trivial in some real life circumstances there might be some clusters which can be depicted by two or more rules namely recessive conjecturable rules for example customers usually prefer to buy inexpensive red wines however on certain occasions such for a birthday celebration they will buy expensive wine therefore we know that there are some people who generally belong to a low value cluster but may simultaneously be assigned to a high value one in this study we propose a new discovery model for mining conjecturable rules to reveal this type of knowledge the experimental results show that our proposed model is able to discover conjecturable rules as well as recessive rules the results of sensitivity analysis are also given for practitioners reference

two topics on linear unequal error protection codes bounds on their length and cyclic code classes it is possible for a linear block code to provide more protection for selected positions in the input message words than is guaranteed by the minimum distance of the code linear codes having this property are called linear unequal error protection luep codes bounds on the length of a luep code that ensures a given unequal error protection are derived a majority decoding method for certain classes of cyclic binary uep codes is treated a list of short i e of length less than 16 binary luep codes of optimal i e minimal length and a list of all cyclic binary uep codes of length less than 40 are included

a graphical technique for determining the number of components in a mixture of normals abstract when a population is assumed to be composed of a finite number of subpopulations a natural model to choose is the finite mixture model it will often be the case however that the number of component distributions is unknown and must be estimated this problem can be difficult for instance the density of two mixed normals is not bimodal unless the means are separated by at least 2 standard deviations hence modality of the data per se can be an insensitive approach to component estimation we demonstrate that a mixture of two normals divided by a normal density having the same mean and variance as the mixed density is always bimodal this analytic result and other related results form the basis for a diagnostic and a test for the number of components in a mixture of normals the density is estimated using a kernel density estimator under the null hypothesis the proposed diagnostic can be approximated by a stationary gaussian process under the alternative hypothesis components in the mixtu

what and when can we gain from the kernel versions of c means algorithm in the recent past different kernelized versions of c means hard and fuzzy clustering algorithms have been proposed here we focus on kernel clustering of only object data x x1 xn rp we first raise a basic question should we really cluster any given object data in the kernel space the answer is no here are our line of arguments 1 the objective of any clustering algorithm is to find natural subgroups in x where the subgroups are defined by a measure of similarity between the vectors in x 2 if we transform the data x into y in another space by a nonlinear transformation and try to find clusters in y then such clusters can be useful if and only if y helps us to find the same clusters that are present in x because that is our objective 3 if y maintains the same structure topology as that of x then the use of y may not give any advantage 4 on the other hand if y changes the structure i e imposes a new structure on the data and that change makes the extraction of the desired clusters present in x easier then clustering of y is useful 5 but when y imposes new nonexistent structures the clustering algorithm may find very strange clusters with no relation to the actual clusters present in x 6 thus when we try to cluster in a transformed space the issue is to know if it could help us to find the clusters present in x to get any benefit from kernel clustering or clustering in any other transformed space we need to answer this question first otherwise we may find completely irrelevant clusters without knowing it and thereby making kernel clustering useless 7 this issue is a philosophical one and is neither dependent on the choice of clustering algorithm nor on the particular transformation kernel function used 8 except for 2 d 3 d data we do not know of any way to answer the question in 6 and for 2 d 3 d data since we can look at the data we do not need kernel clustering therefore there is no benefit from kernel clustering we demonstrate and justify our claims using both synthetic and real datasets with visual assessment as well as with normalized mutual information adjusted rand index and cluster instability we propose to use sammon s nonlinear projection method to get a crude visual representation of the data in the kernel space we discuss the issue of how to choose appropriate parameters of the kernel function but we could not provide a solution to this problem finally we discuss how the kernel parameters and the algorithmic parameters interact

unequal error protection an information theoretic perspective an information theoretic framework for unequal error protection is developed in terms of the exponential error bounds the fundamental difference between the bit wise and message wise unequal error protection uep is demonstrated for fixed length block codes on discrete memoryless channels dmcs without feedback effect of feedback is investigated via variable length block codes it is shown that feedback results in a significant improvement in both bit wise and message wise ueps except the single message case for missed detection the distinction between false alarm and missed detection formalizations for message wise uep is also considered all results presented are at rates close to capacity

hypothesis testing in mixture regression models summary we establish asymptotic theory for both the maximum likelihood and the maximum modified likelihood estimators in mixture regression models moreover under specific and reasonable conditions we show that the optimal convergence rate of n 1 4 for estimating the mixing distribution is achievable for both the maximum likelihood and the maximum modified likelihood estimators we also derive the asymptotic distributions of two log likelihood ratio test statistics for testing homogeneity and we propose a resampling procedure for approximating the p value simulation studies are conducted to investigate the empirical performance of the two test statistics finally two real data sets are analysed to illustrate the application of our theoretical results

information ratios for validating mixture analyses abstract determining the number of components in a mixture of distributions is an important but difficult problem this article introduces a procedure called minimum information ratio estimation and validation mirev which is based on a ratio of fisher information matrices the smallest eigenvalue of the information ratio matrix is used to determine the number of components a measure of uncertainty may be obtained using a bootstrap technique simulations illustrate the effectiveness of the procedure for mixtures of exponential families an expression for the observed information ratio matrix provides insight to the success of the procedure cluster analysis attempts to identify and characterize subpopulations believed to be present in a population a wide variety of methods are available including criterion optimization hierarchical methods and various heuristic methods criterion optimization techniques such as mixture analysis fuzzy clustering and partitioning methods are popular because they