analysis of parameter selection for gustafson kessel fuzzy clustering using jacobian matrix in fuzzy clustering the fuzzy c means fcm is the most known algorithm several extensions and variations of fcm had been proposed in the literature the first important extension to fcm was proposed by gustafson and kessel gk in the gk fuzzy clustering they considered the effect of different cluster shapes except for spherical shapes by replacing the euclidean distance of the fcm objective function with the mahalanobis distance the gk algorithm has become one of the most frequently used clustering algorithms just like fcm the fuzziness index m is a parameter in which the value will greatly influence the performance of the gk algorithm however there is no theoretical work on the parameter selection for the fuzziness index m of gk in this paper we reveal the relation between the stable fixed points of the gk algorithm and the datasets using jacobian matrix analysis and then provide a theoretical base for selecting the fuzziness index m in the gk algorithm some experimental results verify the effectiveness of our theoretical results

general c means clustering model partitional clustering is an important part of cluster analysis based on various theories numerous clustering algorithms have been developed and new clustering algorithms continue to appear in the literature it is known that occam s razor plays a pivotal role in data based models and partitional clustering is categorized as a data based model however no relation had previously been discovered between occam s razor and partitional clustering as we discuss in this paper the three main contributions of this paper can be summarized as follows 1 according to a novel definition of the mean a unifying generative framework for partitional clustering algorithms called a general c means clustering model gcm is presented and studied and 2 based on the local optimality test of the gcm the connection between occam s razor and partitional clustering is established for the first time as its application a comprehensive review of the existing objective function based clustering algorithms is presented based on gcm 3 under a common assumption about partitional clustering a theoretical guide for devising and implementing clustering algorithm is discovered these conclusions are verified by numerical experimental results

the burbea rao and bhattacharyya centroids we study the centroid with respect to the class of information theoretic burbea rao divergences that generalize the celebrated jensen shannon divergence by measuring the non negative jensen difference induced by a strictly convex and differentiable function although those burbea rao divergences are symmetric by construction they are not metric since they fail to satisfy the triangle inequality we first explain how a particular symmetrization of bregman divergences called jensen bregman distances yields exactly those burbea rao divergences we then proceed by defining skew burbea rao divergences and show that skew burbea rao divergences amount in limit cases to compute bregman divergences we then prove that burbea rao centroids can be arbitrarily finely approximated by a generic iterative concave convex optimization algorithm with guaranteed convergence property in the second part of the paper we consider the bhattacharyya distance that is commonly used to measure overlapping degree of probability distributions we show that bhattacharyya distances on members of the same statistical exponential family amount to calculate a burbea rao divergence in disguise thus we get an efficient algorithm for computing the bhattacharyya centroid of a set of parametric distributions belonging to the same exponential families improving over former specialized methods found in the literature that were limited to univariate or diagonal multivariate gaussians to illustrate the performance of our bhattacharyya burbea rao centroid algorithm we present experimental performance results for k means and hierarchical clustering methods of gaussian mixture models

warped k means an algorithm to cluster sequentially distributed data many devices generate large amounts of data that follow some sort of sequentiality e g motion sensors e pens eye trackers etc and often these data need to be compressed for classification storage and or retrieval tasks traditional clustering algorithms can be used for this purpose but unfortunately they do not cope with the sequential information implicitly embedded in such data thus we revisit the well known k means algorithm and provide a general method to properly cluster sequentially distributed data we present warped k means wkm a multi purpose partitional clustering procedure that minimizes the sum of squared error criterion while imposing a hard sequentiality constraint in the classification step we illustrate the properties of wkm in three applications one being the segmentation and classification of human activity wkm outperformed five state of the art clustering techniques to simplify data trajectories achieving a recognition accuracy of near 97 which is an improvement of around 66 over their peers moreover such an improvement came with a reduction in the computational cost of more than one order of magnitude

near optimal regret bounds for reinforcement learning for undiscounted reinforcement learning in markov decision processes mdps we consider the total regret of a learning algorithm with respect to an optimal policy in order to describe the transition structure of an mdp we propose a new parameter an mdp has diameter d if for any pair of states s s there is a policy which moves from s to s in at most d steps on average we present a reinforcement learning algorithm with total regret o ds at after t steps for any unknown mdp with s states a actions per state and diameter d a corresponding lower bound of dsat on the total regret of any learning algorithm is given as well these results are complemented by a sample complexity bound on the number of suboptimal steps taken by our algorithm this bound can be used to achieve a gap dependent regret bound that is logarithmic in t finally we also consider a setting where the mdp is allowed to change a fixed number of l times we present a modification of our algorithm that is able to deal with this setting and show a regret bound of o l1 3t2 3ds a

lower bounds on systolic gossip gossiping is an extensively investigated information dissemination process in which each processor has a distinct item of information and has to collect all the items possessed by the other processors in this paper we provide an innovative and general lower bound technique relying on the novel notion of delay digraph of a gossiping protocol and on the use of matrix norm methods such a technique is very powerful and allows the determination of new and significantly improved lower bounds in many cases in fact we derive the first general lower bound on the gossiping time of systolic protocols i e constituted by a periodic repetition of simple communication steps in particular given any network of n processors and any systolic period s in the directed and the undirected half duplex cases every s systolic gossip protocol takes at least log n log 1 o log log n time steps where is the unique solution between 0 and 1 of p s 2 p s 2 1 with pi 1 2 2i 2 for any integer i 0 we then provide improved lower bounds in the directed and half duplex cases for many well known network topologies such as butterfly de bruijn and kautz graphs all the results are extended also to the full duplex case our technique is very general as for s it allows the determination of improved results even for non systolic protocols in fact for general networks as a simple corollary it yields a lower bound only an o log log n additive factor far from the general one independently proved in proc 1st acm symposium on parallel algorithms and architectures spaa 1989 p 318 topics in combinatorics and graph theory 1990 451 siam journal on computing 21 1 1992 111 discrete applied mathematics 42 1993 75 for all graphs and any non systolic gossip protocol moreover for specific networks it significantly improves with respect to the previously known results even in the full duplex case correspondingly better lower bounds on the gossiping time of non systolic protocols are determined in the directed half duplex and full duplex cases for butterfly de bruijn and kautz graphs even if in this paper we give only a limited number of examples our technique has wide applicability and gives a general framework that often allows to get improved lower bounds on the gossiping time of systolic and non systolic protocols in the directed half duplex and full duplex cases

a validity measure for fuzzy clustering the authors present a fuzzy validity criterion based on a validity function which identifies compact and separate fuzzy c partitions without assumptions as to the number of substructures inherent in the data this function depends on the data set geometric distance measure distance between cluster centroids and more importantly on the fuzzy partition generated by any fuzzy algorithm used the function is mathematically justified via its relationship to a well defined hard clustering validity function the separation index for which the condition of uniqueness has already been established the performance of this validity function compares favorably to that of several others the application of this validity function to color image segmentation in a computer color vision system for recognition of ic wafer defects which are otherwise impossible to detect using gray scale image processing is discussed

gossiping in vertex disjoint paths mode in d dimensional grids and planar graphs abstract this paper continues with the study of the communication modes introduced by j hromkovic r klasing e a stohr dissemination of information in vertex disjoint paths mode part 1 general bounds and gossiping in hypercube like networks technical report no tr ri 93 129 university of paderborn extended abstract presented at wg 93 as a generalization of the standard one way and two way modes allowing messages to be sent between processors of interconnection networks via vertex disjoint paths in one communication step the complexity of communication algorithms is measured by the number of communication steps rounds here the complexity of gossiping in grids and in planar graphs is investigated the main results are the following 1 effective one way and two way gossip algorithms for d dimensional grids d 2 are designed 2 the lower bound 2 log2n log2k log2 log2n 4 is established on the number of rounds of every two way gossip algorithm working on any graph of n nodes and vertex bisection k this proves that the designed two way gossip algorithms on d dimensional grids d 3 are almost optimal and it also shows that the 2 dimensional grid belongs to the best gossip graphs among all planar graphs 3 another lower bound proof is developed to get some tight lower bounds on one way well structured gossip algorithms on planar graphs to the best of the authors knowledge to date all gossip algorithms designed in vertex disjoint paths mode have been well structured

experiments in projection and clustering by simulated annealing simulated annealing is a stochastic relaxation algorithm which has been used successfully to optimize functions of many variables this paper analyzes the simulated annealing algorithm when applied to the minimization of functions from two common problems encountered in exploratory pattern analysis projection and clustering the projection is a nonlinear mapping of patterns in high dimension to two dimensions the simulated annealing mapping is compared to gradient descent minimization of the same objective function as well as eigenvector projection the simulated annealing clustering is compared to a k means algorithm empirical results show that simulated annealing can produce results as good as those obtained by conventional methods but are impractical for small data sets because of the high computational cost simulated annealing does in the case of the mapping problem yield a better optimization and better retained structure for large data sets containing tight gaussian clusters

genie a new fast and outlier resistant hierarchical clustering algorithm abstract the time needed to apply a hierarchical clustering algorithm is most often dominated by the number of computations of a pairwise dissimilarity measure such a constraint for larger data sets puts at a disadvantage the use of all the classical linkage criteria but the single linkage one however it is known that the single linkage clustering algorithm is very sensitive to outliers produces highly skewed dendrograms and therefore usually does not reflect the true underlying data structure unless the clusters are well separated to overcome its limitations we propose a new hierarchical clustering linkage criterion called genie namely our algorithm links two clusters in such a way that a chosen economic inequity measure e g the gini or bonferroni index of the cluster sizes does not increase drastically above a given threshold the presented benchmarks indicate a high practical usefulness of the introduced method it most often outperforms the ward or average linkage in terms of the clustering quality while retaining the single linkage speed the genie algorithm is easily parallelizable and thus may be run on multiple threads to speed up its execution further on its memory overhead is small there is no need to precompute the complete distance matrix to perform the computations in order to obtain a desired clustering it can be applied on arbitrary spaces equipped with a dissimilarity measure e g on real vectors dna or protein sequences images rankings informetric data etc a reference implementation of the algorithm has been included in the open source genie package for r

unsupervised possibilistic clustering in fuzzy clustering the fuzzy c means fcm clustering algorithm is the best known and used method since the fcm memberships do not always explain the degrees of belonging for the data well krishnapuram and keller proposed a possibilistic approach to clustering to correct this weakness of fcm however the performance of krishnapuram and keller s approach depends heavily on the parameters in this paper we propose another possibilistic clustering algorithm pca which is based on the fcm objective function the partition coefficient pc and partition entropy pe validity indexes the resulting membership becomes the exponential function so that it is robust to noise and outliers the parameters in pca can be easily handled also the pca objective function can be considered as a potential function or a mountain function so that the prototypes of pca can be correspondent to the peaks of the estimated function to validate the clustering results obtained through a pca we generalized the validity indexes of fcm this generalization makes each validity index workable in both fuzzy and possibilistic clustering models by combining these generalized validity indexes an unsupervised possibilistic clustering is proposed some numerical examples and real data implementation on the basis of the proposed pca and generalized validity indexes show their effectiveness and accuracy

towards parameter independent data clustering and image segmentation while there are a large amount of clustering algorithms proposed in the literature the clustering results of existing algorithms usually depend on user specified parameters heavily and it is usually difficult to determine the optimal parameters with the pairwise data similarity matrix as the input dominant sets clustering has been shown to be an effective data clustering and image segmentation approach partly due to its ability to find out the underlying data structure and determine the number of clusters automatically however we find that the original dominant sets algorithm is sensitive to the similarity measures used in building the similarity matrix this means that parameter tuning is required to generate satisfactory clustering results and dominant sets clustering results are also parameter dependent in order to remove the dependence on the user specified parameter we study how the similarity measures influence the dominant sets clustering results as a result we propose to transform similarity matrices by histogram equalization before clustering while this transformation is shown to remove the sensitiveness to similarity measures effectively it also results in over segmentation therefore in the next step we present a cluster extension method to overcome the over segmentation effect and generate more reasonable clustering results we test the enhanced clustering algorithm in both data clustering and image segmentation experiments and comparisons with the state of the art algorithms validate the effectiveness of our algorithm highlightswe study how similarity measures influence the dominant sets clustering results we use histogram equalization to remove the dependence on similarity parameters we present a density based cluster extension method to overcome over segmentation experiments validate the effectiveness of our algorithm

a simultaneous learning framework for clustering and classification traditional pattern recognition generally involves two tasks unsupervised clustering and supervised classification when class information is available fusing the advantages of both clustering learning and classification learning into a single framework is an important problem worthy of study to date most algorithms generally treat clustering learning and classification learning in a sequential or two step manner i e first execute clustering learning to explore structures in data and then perform classification learning on top of the obtained structural information however such sequential algorithms cannot always guarantee the simultaneous optimality for both clustering and classification learning in fact the clustering learning in these algorithms just aids the subsequent classification learning and does not benefit from the latter to overcome this problem a simultaneous learning framework for clustering and classification scc is presented in this paper scc aims to achieve three goals 1 acquiring the robust classification and clustering simultaneously 2 designing an effective and transparent classification mechanism 3 revealing the underlying relationship between clusters and classes to this end with the bayesian theory and the cluster posterior probabilities of classes we define a single objective function to which the clustering process is directly embedded by optimizing this objective function the effective and robust clustering and classification results are achieved simultaneously experimental results on both synthetic and real life datasets show that scc achieves promising classification and clustering results at one time

nerf c means non euclidean relational fuzzy clustering abstract the relational fuzzy c means rfcm algorithm can be used to cluster a set of n objects described by pair wise dissimilarity values if and only if there exist n points in r n 1 whose squared euclidean distances precisely match the given dissimilarity data this strong restriction on the dissimilarity data renders rfcm inapplicable to most relational clustering problems this paper substantially improves rfcm by generalizing it to the case of arbitrary symmetric dissimilarity data the generalization is obtained using a computationally efficient modification of the existing algorithm that is equivalent to applying a spreading transformation to the dissimilarity data while the method given applies specifically to dissimilarity data a simple transformation can be used to convert similarity relations into dissimilarity data so the method is applicable to any numerical relational data that are positive reflexive or anti reflexive and symmetric numerical examples illustrate and compare the present approach to problems that can be studied with alternatives such as the linkage algorithms

linearly parameterized bandits we consider bandit problems involving a large possibly infinite collection of arms in which the expected reward of each arm is a linear function of an r dimensional random vector z rr where r 2 the objective is to minimize the cumulative regret and bayes risk when the set of arms corresponds to the unit sphere we prove that the regret and bayes risk is of order r t by establishing a lower bound for an arbitrary policy and showing that a matching upper bound is obtained through a policy that alternates between exploration and exploitation phases the phase based policy is also shown to be effective if the set of arms satisfies a strong convexity condition for the case of a general set of arms we describe a near optimal policy whose regret and bayes risk admit upper bounds of the form o r t log3 2t

dynamic pricing with multiple products and partially specified demand distribution we study a dynamic pricing problem with multiple products and infinite inventories the demand for these products depends on the selling prices and on parameters unknown to the seller their value can be learned from accumulating sales data using statistical estimation techniques the quality of the parameter estimates is influenced by the amount of price dispersion however a large amount of variation in the selling prices can be costly since it means that suboptimal prices are used the seller thus needs to balance optimizing the quality of the parameter estimates and optimizing instant revenue i e exploitation and exploration in this study we propose a pricing policy for this dynamic pricing problem the key idea is to use at each time period the price that is optimal with respect to current parameter estimates with an additional constraint that ensures sufficient price dispersion we measure the price dispersion by the smallest eigenvalue of the design matrix and show how a desired growth rate of this eigenvalue can be achieved by a simple quadratic constraint in the price optimization problem we study the performance of our pricing policy by providing bounds on the regret which measures the expected revenue loss caused by using suboptimal prices

sided and symmetrized bregman centroids in this paper we generalize the notions of centroids and barycenters to the broad class of information theoretic distortion measures called bregman divergences bregman divergences form a rich and versatile family of distances that unifies quadratic euclidean distances with various well known statistical entropic measures since besides the squared euclidean distance bregman divergences are asymmetric we consider the left sided and right sided centroids and the symmetrized centroids as minimizers of average bregman distortions we prove that all three centroids are unique and give closed form solutions for the sided centroids that are generalized means furthermore we design a provably fast and efficient arbitrary close approximation algorithm for the symmetrized centroid based on its exact geometric characterization the geometric approximation algorithm requires only to walk on a geodesic linking the two left right sided centroids we report on our implementation for computing entropic centers of image histogram clusters and entropic centers of multivariate normal distributions that are useful operations for processing multimedia information and retrieval these experiments illustrate that our generic methods compare favorably with former limited ad hoc methods

density peak clustering based on relative density relationship abstract the density peak clustering algorithm treats local density peaks as cluster centers and groups non center data points by assuming that one data point and its nearest higher density neighbor are in the same cluster while this algorithm is shown to be promising in some applications its clustering results are found to be sensitive to density kernels and large density differences across clusters tend to result in wrong cluster centers in this paper we attribute these problems to the inconsistency between the assumption and implementation adopted in this algorithm while the assumption is based totally on relative density relationship this algorithm adopts absolute density as one criterion to identify cluster centers this observation prompts us to present a cluster center identification criterion based only on relative density relationship specifically we define the concept of subordinate to describe the relative density relationship and use the number of subordinates as a criterion to identify cluster centers our approach makes use of only relative density relationship and is less influenced by density kernels and density differences across clusters in addition we discuss the problems of two existing density kernels and present an average distance based kernel in data clustering experiments we validate the new criterion and density kernel respectively and then test the whole algorithm and compare with some other clustering algorithms

relative entropy exponential utility and robust dynamic pricing in the area of dynamic revenue management optimal pricing policies are typically computed on the basis of an underlying demand rate model from the perspective of applications this approach implicitly assumes that the model is an accurate representation of the real world demand process and that the parameters characterizing this model can be accurately calibrated using data in many situations neither of these conditions are satisfied indeed models are usually simplified for the purpose of tractability and may be difficult to calibrate because of a lack of data moreover pricing policies that are computed under the assumption that the model is correct may perform badly when this is not the case this paper presents an approach to single product dynamic revenue management that accounts for errors in the underlying model at the optimization stage uncertainty in the demand rate model is represented using the notion of relative entropy and a tractable reformulation of the robust pricing problem is obtained using results concerning the change of probability measure for point processes the optimal pricing policy is obtained through a version of the so called isaacs equation for stochastic differential games and the structural properties of the optimal solution are obtained through an analysis of this equation in particular i closed form solutions for the special case of an exponential nominal demand rate model ii general conditions for the exchange of the max and the min in the differential game and iii the equivalence between the robust pricing problem and that of single product revenue management with an exponential utility function without model uncertainty are established through the analysis of this equation

unsupervised segmentation of markov random field modeled textured images using selectionist relaxation among the existing texture segmentation methods those relying on markov random fields have retained substantial interest and have proved to be very efficient in supervised mode the use of markov random fields in unsupervised mode is however hampered by the parameter estimation problem the recent solutions proposed to overcome this difficulty rely on assumptions about the shapes of the textured regions or about the number of textures in the input image that may not be satisfied in practice in this paper an evolutionary approach selectionist relaxation is proposed as a solution to the problem of segmenting markov random field modeled textures in unsupervised mode in selectionist relaxation the computation is distributed among a population of units that iteratively evolves according to simple and local evolutionary rules a unit is an association between a label and a texture parameter vector the units whose likelihood is high are allowed to spread over the image and to replace the units that receive lower support from the data consequently some labels are growing while others are eliminated starting with an initial random population this evolutionary process eventually results in a stable labelization of the image which is taken as the segmentation in this work the generalized ising model is used to represent textured data because of the awkward nature of the partition function in this model a high temperature approximation is introduced to allow the evaluation of unit likelihoods experimental results on images containing various synthetic and natural textures are reported

mblda a novel multiple between class linear discriminant analysis abstract linear discriminant analysis lda with its extensions is a group of classical methods in dimensionality reduction for supervised learning however when some classes are far away from the others it may be difficult to find the optimal direction by lda because of the average between class scatter moreover ldas are always time consuming for high dimensional problem since the involved generalized eigenvalue problem is needed to be solved in this paper a multiple between class linear discriminant analysis mblda is proposed for dimensionality reduction mblda finds the transformation directions by approximating the solution to a min max programming problem leading to well separability in the reduced space with a fast learning speed on the high dimensional problem it is proved theoretically that the proposed method can deal with the special generalized eigenvalue problem by solving a underdetermined homogeneous system of linear equations experimental results on the artificial and benchmark datasets show that mblda can not only reduce the dimension to a powerful linear discriminant level but also have a fast learning speed

a contribution to convergence theory of fuzzy c means and derivatives in this paper we revisit the convergence and optimization properties of fuzzy clustering algorithms in general and the fuzzy c means fcm algorithm in particular our investigation includes probabilistic and a slightly modified implementation of possibilistic memberships which will be discussed under a unified view we give a convergence proof for the axis parallel variant of the algorithm by gustafson and kessel that can be generalized to other algorithms more easily than in the usual approach using reformulated fuzzy clustering algorithms we apply banach s classical contraction principle and establish a relationship between saddle points and attractive fixed points for the special case of fcm we derive a sufficient condition for fixed points to be attractive allowing identification of them as local minima of the objective function excluding the possibility of a saddle point

convergence of the single pass and online fuzzy c means algorithms scalable versions of the widely used fuzzy c means clustering algorithm called single pass fuzzy c means and online fuzzy c means have been recently introduced both algorithms facilitate scaling to very large numbers of examples while providing partitions that very closely approximate those one would obtain using fuzzy c means both algorithms have been successfully applied to a number of datasets most notably magnetic resonance image volumes of the human brain in this letter we show that weighting examples in the fuzzy c means algorithm does not cause a violation in its convergence proof and we provide a separate proof of convergence that holds for any dataset

updating the inverse of a matrix the sherman morrison woodbury formulas relate the inverse of a matrix after a small rank perturbation to the inverse of the original matrix the history of these fomulas is presented and various applications to statistics networks structural analysis asymptotic analysis optimization and partial differential equations are discussed the sherman morrison woodbury formulas express the inverse of a matrix after a small rank perturbation in terms of the inverse of the original matrix this paper surveys the history of these formulas and we examine some applications where these formulas are helpful

a maximum variance cluster algorithm we present a partitional cluster algorithm that minimizes the sum of squared error criterion while imposing a hard constraint on the cluster variance conceptually hypothesized clusters act in parallel and cooperate with their neighboring clusters in order to minimize the criterion and to satisfy the variance constraint in order to enable the demarcation of the cluster neighborhood without crucial parameters we introduce the notion of foreign cluster samples finally we demonstrate a new method for cluster tendency assessment based on varying the variance constraint parameter

a redundancy based measure of dissimilarity among probability distributions for hierarchical clustering criteria we introduce novel dissimilarity into a probabilistic clustering task to properly measure dissimilarity among multiple clusters when each cluster is characterized by a subpopulation in the mixture model this measure of dissimilarity is called redundancy based dissimilarity among probability distributions from aspects of both source coding and a statistical hypothesis test we shed light on several of the theoretical reasons for the redundancy based dissimilarity among probability distributions being a reasonable measure of dissimilarity among clusters we also elucidate a principle in common for the measures of redundancy based dissimilarity and ward s method in terms of hierarchical clustering criteria moreover we show several related theorems that are significant for clustering tasks in the experiments properties of the measure of redundancy based dissimilarity are examined in comparison with several other measures

ncm neutrosophic c means clustering algorithm abstract in this paper a new clustering algorithm neutrosophic c means ncm is introduced for uncertain data clustering which is inspired from fuzzy c means and the neutrosophic set framework to derive such a structure a novel suitable objective function is defined and minimized and the clustering problem is formulated as a constrained minimization problem whose solution depends on the objective function in the objective function two new types of rejection have been introduced the ambiguity rejection which concerns the patterns lying near the cluster boundaries and the distance rejection dealing with patterns that are far away from all the clusters these measures are able to manage uncertainty due to imprecise and or incomplete definition of the clusters we conducted several experiments with synthetic and real data sets the results are encouraging and compared favorably with results from other methods as fcm pcm and fpcm algorithms on the same data sets finally the proposed method was applied into image segmentation algorithm the experimental results show that the proposed algorithm can be considered as a promising tool for data clustering and image processing

fuzzy order statistics and their application to fuzzy clustering the median and the median absolute deviation mad are robust statistics based on order statistics order statistics are extended to fuzzy sets to define a fuzzy median and a fuzzy mad the fuzzy c means fcm clustering algorithm is defined for any p norm pfcm including the l sub 1 norm 1fcm the 1fcm clustering algorithm is implemented via the alternating optimization ao method and the clustering centers are shown to be the fuzzy median the resulting ao 1fcm clustering algorithm is called the fuzzy c medians fcmed clustering algorithm an example illustrates the robustness of the fcmed

baum katz nagaev type results for martingales abstract we obtain a baum katz nagaev type theorem for bounded martingale difference sequences that have more than a second moment and prove that the celebrated hsu robbins erd s theorem fails for martingales

optimization of clustering criteria by reformulation various hard fuzzy and possibilistic clustering criteria objective functions are useful as bases for a variety of pattern recognition problems at present many of these criteria have customized individual optimization algorithms because of the specialized nature of these algorithms experimentation with new and existing criteria can be very inconvenient and costly in terms of development and implementation time this paper shows how to reformulate some clustering criteria so that specialized algorithms can be replaced by general optimization routines found in commercially available software we prove that the original and reformulated versions of each criterion are fully equivalent finally two numerical examples are given to illustrate reformulation

improved possibilistic c means clustering algorithms a possibilistic approach was proposed in a previous paper for c means clustering and two algorithms realizing this approach were reported in two previous papers although the possibilistic approach is sound these two algorithms tend to find identical clusters in this paper we modify and improve these algorithms to overcome their shortcoming the numerical results demonstrate that the improved algorithms can determine proper clusters and they can realize the advantages of the possibilistic approach

a new fuzzy clustering validity index with a median factor for centroid based clustering determining the number of clusters which is usually approved by domain experts or evaluated by clustering validity indexes is an important issue in clustering analysis this study discusses the effectiveness of clustering validity indexes for centroid based partitional clustering algorithms most general purpose clustering validity indexes take the minimum maximum distance between a pair of data objects a pair of cluster centroids or an object and a centroid as an important evaluation factor however they may present unstable results especially when two centroids are allocated closely to alleviate this problem a new clustering validity index which is termed the wu and li index wli is proposed in this paper our proposed wli partially allows to some extent the existence of closely allocated centroids in the clustering results by considering not only the minimum but the median distances between a pair of centroids as well therefore possessing better stability the performances of wli and some existing clustering validity indexes are evaluated and compared by running the fuzzy c means algorithm for clustering various types of datasets including artificial datasets uci datasets and images experimental results have shown that wli has the more accurate and satisfactory performance than other indexes

axiomatic generalization of the membership degree weighting function for fuzzy c means clustering for decades practitioners have been using the center based partitional clustering algorithms like fuzzy c means fcm which rely on minimizing an objective function comprising of an appropriately weighted sum of distances of each data point from the cluster representatives numerous generalizations in terms of choice of the distance function have been introduced for fcm since its inception in a stark contrast to this fact to the best of our knowledge there has not been any significant effort to address the issue of convergence of the algorithm under the structured generalization of the weighting function here by generalization we mean replacing the conventional weighting function uijm where uij indicates the membership of data xi to cluster cj and m is the real valued fuzzifier with 1 m with a more general function g uij which can assume a wide variety of forms under some mild assumptions in this article for the first time we present a novel axiomatic development of the general weighting function based on the membership degrees we formulate the fuzzy clustering problem along with the intuitive justification of the technicalities behind the axiomatic approach we develop an alternative optimization ao procedure to solve the main clustering problem by solving the partial optimization problems in an iterative way the novelty of the article lies in the development of an axiomatic generalization of the weighting function of fcm formulation of an ao algorithm for the fuzzy clustering problem with the extension to this general class of weighting functions and a detailed convergence analysis of the proposed algorithm

a novel adaptive possibilistic clustering algorithm in this paper a novel possibilistic c means clustering algorithm called adaptive possibilistic c means is presented its main feature is that its parameters after their initialization are properly adapted during its execution provided that the algorithm starts with a reasonable overestimate of the number of physical clusters formed by the data it is capable in principle to unravel them a long standing issue in the clustering literature this is due to the fully adaptive nature of the proposed algorithm that enables the removal of the clusters that gradually become obsolete in addition the adaptation of all its parameters increases the flexibility of the algorithm in following the variations in the formation of the clusters that occur from iteration to iteration theoretical results that are indicative of the convergence behavior of the algorithm are also provided finally extensive simulation results on both synthetic and real data highlight the effectiveness of the proposed algorithm

a unified continuous optimization framework for center based clustering methods center based partitioning clustering algorithms rely on minimizing an appropriately formulated objective function and different formulations suggest different possible algorithms in this paper we start with the standard nonconvex and nonsmooth formulation of the partitioning clustering problem we demonstrate that within this elementary formulation convex analysis tools and optimization theory provide a unifying language and framework to design analyze and extend hard and soft center based clustering algorithms through a generic algorithm which retains the computational simplicity of the popular k means scheme we show that several well known and more recent center based clustering algorithms which have been derived either heuristically or and have emerged from intuitive analogies in physics statistical techniques and information theoretic perspectives can be recovered as special cases of the proposed analysis and we streamline their relationships

a possibilistic fuzzy c means clustering algorithm in 1997 we proposed the fuzzy possibilistic c means fpcm model and algorithm that generated both membership and typicality values when clustering unlabeled data fpcm constrains the typicality values so that the sum over all data points of typicalities to a cluster is one the row sum constraint produces unrealistic typicality values for large data sets in this paper we propose a new model called possibilistic fuzzy c means pfcm model pfcm produces memberships and possibilities simultaneously along with the usual point prototypes or cluster centers for each cluster pfcm is a hybridization of possibilistic c means pcm and fuzzy c means fcm that often avoids various problems of pcm fcm and fpcm pfcm solves the noise sensitivity defect of fcm overcomes the coincident clusters problem of pcm and eliminates the row sum constraints of fpcm we derive the first order necessary conditions for extrema of the pfcm objective function and use them as the basis for a standard alternating optimization approach to finding local minima of the pfcm objective functional several numerical examples are given that compare fcm and pcm to pfcm our examples show that pfcm compares favorably to both of the previous models since pfcm prototypes are less sensitive to outliers and can avoid coincident clusters pfcm is a strong candidate for fuzzy rule based system identification

fuzzy consensus clustering with applications on big data consensus clustering aims to find a single partition of data that agrees as much as possible with existing basic partitions given its robustness and generalizability consensus clustering has emerged as a promising solution to find cluster structures inside heterogeneous big data rising from various application domains in the area of fuzzy systems however research along this line is still in its initial stage with some unsystematic algorithmic studies finding a fuzzy consensus partition from multiple fuzzy basic partitions in an efficient flexible and robust way is still an exciting open problem calling for further investigation in light of this this paper provides a systematic study of fuzzy consensus clustering fcc from a utility perspective specifically we first define the objective function of fcc clearly using the novel fuzzified contingency matrix we then derive a family of fcc utility functions termed as fccu that can transform fcc to a weighted piecewise fuzzy c means clustering pifcm problem this helps us to establish an algorithmic framework for fcc with flexible choice of utility functions and speeds fcc significantly with a fcm like iterative process of pifcm to meet the big data challenge we further parallelize fcc on the spark platform with both vertical and horizontal segmentation schemes extensive experiments on various real world datasets demonstrate the excellent performance of fcc even with a majority of poor basic partitions in particular our method exhibits interesting potential for big data clustering in two real life applications concerned with online event detection and overlapping community detection respectively

fast and robust fuzzy c means clustering algorithms incorporating local information for image segmentation fuzzy c means fcm algorithms with spatial constraints fcm s have been proven effective for image segmentation however they still have the following disadvantages 1 although the introduction of local spatial information to the corresponding objective functions enhances their insensitiveness to noise to some extent they still lack enough robustness to noise and outliers especially in absence of prior knowledge of the noise 2 in their objective functions there exists a crucial parameter a used to balance between robustness to noise and effectiveness of preserving the details of the image it is selected generally through experience and 3 the time of segmenting an image is dependent on the image size and hence the larger the size of the image the more the segmentation time in this paper by incorporating local spatial and gray information together a novel fast and robust fcm framework for image segmentation i e fast generalized fuzzy c means fgfcm clustering algorithms is proposed fgfcm can mitigate the disadvantages of fcm s and at the same time enhances the clustering performance furthermore fgfcm not only includes many existing algorithms such as fast fcm and enhanced fcm as its special cases but also can derive other new algorithms such as fgfcm s1 and fgfcm s2 proposed in the rest of this paper the major characteristics of fgfcm are 1 to use a new factor s i j as a local both spatial and gray similarity measure aiming to guarantee both noise immunity and detail preserving for image and meanwhile remove the empirically adjusted parameter a 2 fast clustering or segmenting image the segmenting time is only dependent on the number of the gray levels q rather than the size n q of the image and consequently its computational complexity is reduced from o nci 1 to o qci 2 where c is the number of the clusters i 1 and i 2

a multi prototype clustering algorithm clustering is an important unsupervised learning technique widely used to discover the inherent structure of a given data set some existing clustering algorithms uses single prototype to represent each cluster which may not adequately model the clusters of arbitrary shape and size and hence limit the clustering performance on complex data structure this paper proposes a clustering algorithm to represent one cluster by multiple prototypes the squared error clustering is used to produce a number of prototypes to locate the regions of high density because of its low computational cost and yet good performance a separation measure is proposed to evaluate how well two prototypes are separated multiple prototypes with small separations are grouped into a given number of clusters in the agglomerative method new prototypes are iteratively added to improve the poor cluster separations as a result the proposed algorithm can discover the clusters of complex structure with robustness to initial settings experimental results on both synthetic and real data sets demonstrate the effectiveness of the proposed clustering algorithm

numerical conditioning problems and solutions for nonparametric i i d statistical active contours in this paper we propose an active contour model based on nonparametric independent and identically distributed i i d statistics of the image that can segment an image without any a priori information about the intensity distributions of the region of interest or the background this is not however the first active contour model proposed to solve the segmentation problem under these same assumptions in contrast to prior active contour models based on nonparametric i i d statistics we do not formulate our optimization criterion according to any distance measure between estimated probability densities inside and outside the active contour instead treating the segmentation problem as a pixel wise classification problem we formulate an active contour to minimize the unbiased pixel wise average misclassification probability amp this not only simplifies the problem by avoiding the need to arbitrarily select among many sensible distance measures to measure the difference between the probability densities estimated inside and outside the active contour but it also solves a numerical conditioning problem that arises with such prior active contour models as a result the amp model exhibits faster convergence with higher accuracy and robustness when compared to active contour models previously formulated to solve the same nonparametric i i d statistical segmentation problem via probability distances to discuss this improved numerical behavior more precisely we introduce the notion of x201c conditioning ratio x201d and demonstrate that the proposed amp active contour is numerically better conditioned i e exhibits a much smaller conditioning ratio than prior probability distance based active contours

generalized fuzzy c means clustering strategies using l sub p norm distances fuzzy c means fcm is a useful clustering technique modifications of fcm using l sub 1 norm distances increase robustness to outliers object and relational data versions of fcm clustering are defined for the more general case where the l sub p norm p spl ges 1 or semi norm 0 0 in order to facilitate the empirical examination of the object data models both object and relational approaches are included in a numerical study

the possibilistic c means algorithm insights and recommendations recently the possibilistic c means algorithm pcm was proposed to address the drawbacks associated with the constrained memberships used in algorithms such as the fuzzy c means fcm in this issue barni et al 1996 report a difficulty they faced while applying the pcm and note that it exhibits an undesirable tendency to converge to coincidental clusters the purpose of this paper is not just to address the issues raised by barni et al but to go further and analytically examines the underlying principles of the pcm and the possibilistic approach in general we analyze the data sets used by barni et al and interpret the results reported by them in the light of our findings

deterministic annealing gustafson kessel fuzzy clustering algorithm abstract the gustafson and kessel gk fuzzy clustering algorithm proposed by gustafson and kessel in 1979 was the first important extension to the fuzzy c means fcm algorithm up to now the gk algorithm had become one of the most commonly used fuzzy clustering algorithms where the mahalanobis distance is used as a dissimilarity measure to provide more effectiveness and robustness than the fcm algorithm recently chaomurilige et al 2015 proposed a theoretical analysis on the parameter selection for the gk algorithm in which they indicated that the parameter of the fuzziness index heavily influences the performance of the gk algorithm in this paper we propose a novel gk fuzzy clustering algorithm based on the deterministic annealing approach for decreasing the effect of parameters we first consider maximizing the shannon s entropy of membership functions to the gk objective function and then use deterministic annealing to adjust the annealing parameter we also mathematically provide a theoretical initialization lower bound for the annealing parameter of the proposed deterministic annealing gk da gk algorithm comparisons between the da gk algorithm and other methods are made the computational complexity of the proposed method is also provided experimental results and comparisons actually verify theoretical results and also indicate the superiority and effectiveness of the proposed da gk algorithm

on the optimality of conditional expectation as a bregman predictor we consider the problem of predicting a random variable x from observations denoted by a random variable z it is well known that the conditional expectation e x z is the optimal l sup 2 predictor also known as the least mean square error predictor of x among all borel measurable functions of z in this orrespondence we provide necessary and sufficient conditions for the general loss functions under which the conditional expectation is the unique optimal predictor we show that e x z is the optimal predictor for all bregman loss functions blfs of which the l sup 2 loss function is a special case moreover under mild conditions we show that the blfs are exhaustive i e if for every random variable x the infimum of e f x y over all constants y is attained by the expectation e x then f is a blf

quasi likelihood and generalizing the em algorithm this paper is concerned with situations in which there are missing or otherwise incomplete data and the full likelihood may not be available extensions of the em algorithm are developed to deal with estimation via general estimating functions and in particular the quasi score the e step is replaced by projecting the quasi score and the m step requires the solution of an estimating equation the standard em algorithm can be obtained as a particular case if the likelihood is available

observer biased fuzzy clustering as generated by clustering algorithms clusterings or partitions are hypotheses on data explanation which are better evaluated by experts from the application domain in general clustering algorithms allow a limited usage of domain knowledge about the cluster formation process in this study we propose both a design technique and a new partitioning based clustering algorithm which can be used to assist the data analyst while looking for a set of meaningful clusters i e clusters that actually correspond to the underlying data structure following an observer metaphor according to which the perception of a group of objects depends on the observer position the closer an observer is from an image more details s he perceives we resort to shrinkage to incorporate a regularization term accounting for the observation point within the objective function of an otherwise unbiased clustering algorithm this technique allows our resulting biased algorithm to generate a set of reasonable partitions i e partitions validated by a given cluster validity index corresponding to views of data with different levels of granularity levels of detail in different regions of the data space for the illustration of the design technique we adopted the fuzzy c means fcm algorithm as the unbiased clustering algorithm and include a convergence theorem assuring that changing the point of observation in the corresponding biased algorithm fcm with focal point fcmfp does not jeopardize its convergence experimental studies on both synthetic and real data are included to illustrate the usefulness of the approach in addition and as a convenient side effect of using shrinkage the experimental results suggest that our biased algorithm fcmfp not only seems to scale better than the successive runs of the unbiased one fcm but on the average seems to produce clusters exhibiting higher validity index values as well in addition less sensitivity to initialization was observed for the biased algorithm when compared with the unbiased one

on the unification of possibilistic fuzzy clustering axiomatic development and convergence analysis abstract fuzzy partitional clustering algorithms like fuzzy c means fcm are sensitive to noise points or outliers because of the probabilistic constraint the sum of the membership values of a specific data point to all the clusters is 1 which enables it to classify any noise or outlier in a specific cluster this not only hampers the clustering performance corresponding to that specific point but also leaves a huge impact on the overall clustering process by deviating the cluster centroids through a significant amount in order to improve this weakness possibilistic approach relaxes the probabilistic constraints nevertheless due to lack of constraints imposed on the typicality matrix depending on initialization possibilistic clustering algorithms suffer from coincident clusters because their membership expressions do not consider the distance to other cluster representatives fuzzy possibilistic clustering which puts a linear constraint on the sum of all the typicality values takes care of this problem but if the number of data points is high the generated typicality values are very small due to the linear constraint on their sums in order to do away with that problem the possibilistic fuzzy clustering algorithm was developed in this article we present an axiomatic development and extension of the possibilistic fuzzy clustering algorithm in three directions choice of the dissimilarity measure joint contribution function and the penalty function we provide a thorough convergence analysis of the proposed generalized possibilistic fuzzy clustering algorithm we investigate the relationships of the proposed generalization with the existing variations of the possibilistic c means pcm fuzzy possiblistic c means fpcm and possibilistic fuzzy c means pfcm algorithms in literature to the best of our knowledge this is the first article of its kind to provide a unification to the long list of possibilistic fuzzy possiblistic and possibilistic fuzzy clustering methods

a simulated annealing algorithm for the clustering problem abstract in this paper we discuss the solution of the clustering problem usually solved by the k means algorithm the problem is known to have local minimum solutions which are usually what the k means algorithm obtains the simulated annealing approach for solving optimization problems is described and is proposed for solving the clustering problem the parameters of the algorithm are discussed in detail and it is shown that the algorithm converges to a global solution of the clustering problem we also find optimal parameters values for a specific class of data sets and give recommendations on the choice of parameters for general data sets finally advantages and disadvantages of the approach are presented

comments on a possibilistic approach to clustering in this comment we report a difficulty with the application of the possibilistic approach to fuzzy clustering pcm proposed by keller and krishnapuram 1993 in applying this algorithm we found that it has the undesirable tendency to produce coincidental clusters results illustrating this tendency are reported and a possible explanation for the pcm behavior is suggested

a clustering ensemble the aim of clustering ensemble is to combine multiple base partitions into a robust stable and accurate partition one of the key problems of clustering ensemble is how to exploit the cluster structure information in each base partition evidence accumulation is an effective framework which can convert the base partitions into a co association matrix this matrix describes the frequency of a pair of points partitioned into the same cluster but ignores some hidden information in the base partitions in this paper we reveal some of those information by refining the co association matrix from data point and base cluster level from the data point level as pairs of points in the same base cluster may have varied similarities their contributions to the co association matrix can be different from the cluster level since the base clusters may have diversified qualities the contribution of a base cluster as a whole can also be different from those of others after being refined the co association matrix is transformed into a path based similarity matrix so that more global information of the cluster structure is incorporated into the matrix finally spectral clustering is applied to the matrix to generate the final clustering result experimental results on 8 synthetic and 8 real data sets demonstrate that the clustering ensemble based on the refined co association matrix outperforms some state of the art clustering ensemble schemes highlightsa two level refined co association matrix for cluster ensemble is proposed the refined co association matrix is transformed by path based measure a theoretical background of the refinement is given the proposed method outperforms some state of the art ensemble methods

soft transition from probabilistic to possibilistic fuzzy clustering in the fuzzy clustering literature two main types of membership are usually considered a relative type termed probabilistic and an absolute or possibilistic type indicating the strength of the attribution to any cluster independent from the rest there are works addressing the unification of the two schemes here we focus on providing a model for the transition from one schema to the other to exploit the dual information given by the two schemes and to add flexibility for the interpretation of results we apply an uncertainty model based on interval values to memberships in the clustering framework obtaining a framework that we term graded possibility we outline a basic example of graded possibilistic clustering algorithm and add some practical remarks about its implementation the experimental demonstrations presented highlight the different properties attainable through appropriate implementation of a suitable graded possibilistic model an interesting application is found in automated segmentation of diagnostic medical images where the model provides an interactive visualization tool for this task

a new convergence proof of fuzzy c means in this letter we give a new more direct derivation of the convergence properties of the fuzzy c means fcm algorithm using the equivalence between the original and reduced fcm criterion from the point of view of the reduced criterion the fcm algorithm is simply a steepest descent algorithm with variable steplength we prove that steplength adjustment follows from the majorization principle for steplength by applying the majorization principle we give a straightforward proof of global convergence further convergence properties follow immediately using known results of optimization theory

divergence measures based on the shannon entropy a novel class of information theoretic divergence measures based on the shannon entropy is introduced unlike the well known kullback divergences the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved more importantly their close relationship with the variational distance and the probability of misclassification error are established in terms of bounds these bounds are crucial in many applications of divergence measures the measures are also well characterized by the properties of nonnegativity finiteness semiboundedness and boundedness

fuzzy clustering of fuzzy data based on robust loss functions and ordered weighted averaging abstract in many real cases the data are not expressed in term of single values but are imprecise in all these cases standard clustering methods for single valued data are unable to properly take into account the imprecise nature of the data in this paper by considering the partitioning around medoids pam approach in a fuzzy framework we propose a fuzzy clustering method for imprecise data formalized in a fuzzy manner in particular in order to neutralize the negative effects of possible outlier fuzzy data in the clustering process we proposed a robust fuzzy c medoids clustering method for fuzzy data based on the combination of huber s m estimators and yager s owa ordered weighted averaging operators the proposed method is able to smooth the influence of anomalous data by means of a suitable parameter the so called typicality parameter capable to tune the influence of the outliers the performance of the proposed method has been shown by means of a simulation study composed of experiments on i simple two dimensional dataset ii benchmark datasets and iii the fuzzy art outliers dataset the comparison made with the robust clustering methods known from the literature indicates the competitiveness of the introduced method to others an application of the suggested method to a real dataset is also provided and the results of the method has been compared with other clustering methods suggested in the literature in the application the comparative assessment has shown the informational gain in term of additional information of the proposed method vs the other robust methods

an extension to possibilistic fuzzy cluster analysis we explore an approach to possibilistic fuzzy clustering that avoids a severe drawback of the conventional approach namely that the objective function is truly minimized only if all cluster centers are identical our approach is based on the idea that this undesired property can be avoided if we introduce a mutual repulsion of the clusters so that they are forced away from each other we develop this approach for the possibilistic fuzzy c means algorithm and the possibilistic gustafson kessel algorithm in our experiments we found that in this way we can combine the partitioning property of the probabilistic fuzzy c means algorithm with the advantages of a possibilistic approach w r t the interpretation of the membership degrees

on the convergence of least squares estimates in white noise the problem of convergence of least squares ls estimates in a stochastic linear regression model with white noise is considered it is well known that if the parameter estimates are known to converge the convergence analysis for many adaptive systems can be rendered considerably less arduous for an important case where the regression vector is a measurable function of the observations and the noise is gaussian it has been shown by using a bayesian embedding argument that the ls estimates converge almost surely for almost all true parameters in the parameter space except for a zero measure set however nothing can be said about a particular given system which is usually the objective it has long been conjectured that such a bad zero measure set in the parameter space does not actually exist a conclusive answer to this important question is provided and it is shown that the set can indeed exist this then shows that to provide conclusive convergence results for stochastic adaptive systems it is necessary to resort to a sample pathwise analysis instead of the bayesian embedding approach

border peeling clustering in this paper we present a novel non parametric clustering technique our technique is based on the notion that each latent cluster is comprised of layers that surround its core where the external layers or border points implicitly separate the clusters unlike previous techniques such as dbscan where the cores of the clusters are defined directly by their densities here the latent cores are revealed by a progressive peeling of the border points analyzing the density of the local neighborhoods allows identifying the border points and associating them with points of inner layers we show that the peeling process adapts to the local densities and characteristics to successfully separate adjacent clusters of possibly different densities we extensively tested our technique on large sets of labeled data including high dimensional datasets of deep features that were trained by a convolutional neural network we show that our technique is competitive to other state of the art non parametric methods using a fixed set of parameters throughout the experiments

multistage branch and bound merging for planar surface segmentation in disparity space an iterative split and merge framework for the segmentation of planar surfaces in the disparity space is presented disparity of a scene is modeled by approximating various surfaces in the scene to be planar in the split phase the number of planar surfaces along with the underlying plane parameters is assumed to be known from the initialization or from the previous merge phase based on these parameters planar surfaces in the disparity image are labeled to minimize the residuals between the actual disparity and the modeled disparity the labeled planar surfaces are separated into spatially continuous regions which are treated as candidates for the merging that follows the regions are merged together under a maximum variance constraint while maximizing the merged area a multistage branch and bound algorithm is proposed to carry out this optimization efficiently each stage of the branch and bound algorithm separates a planar surface from the set of spatially continuous regions the multistage merging estimates the number of planar surfaces and their labeling the splitting and the multistage merging is repeated till convergence is reached or satisfactory results are achieved experimental results are presented for variety of stereo image data

extending information theoretic validity indices for fuzzy clustering previously eight popular information theoretic based cluster validity indices have been generalized and tested for probabilistic partitions built by the expectation maximization em algorithm for the gaussian mixture model however the analysis was limited to probabilistic clusters and there were limited explanations for differences in the performance of the indices in this paper we extend the tests to partitions found by fuzzy c means fcm and provide further explanations and insights about the performance of these indices of the eight generalized indices we advocate a normalized version of the soft mutual information cluster validity index nmi sm as the best overall choice as it outperforms the other seven indices for both fcm and em according to our tests on synthetic and real data the superiority of nmism is most pronounced for datasets with overlapped and or varying sized clusters finally we provide a theoretical analysis which helps explain the superior performance of nmism compared with the other three normalizations of soft mutual information

a generalization of distance functions for fuzzy c means clustering with centroids of arithmetic means fuzzy c means fcm is a widely used fuzzy clustering method which allows an object to belong to two or more clusters with a membership grade between zero and one despite the considerable efforts made by the clustering community the common characteristics of distance functions suitable for fcm remain unclear to fill this crucial void in this paper we first provide a generalized definition of distance functions that fit fcm directly the goal is to provide more flexibility to fcm in the choice of distance functions while preserving the simplicity of fcm by using the centroids of arithmetic means indeed we show that any distance function that fits fcm directly can be derived by a continuously differentiable convex function and thus is an instance of the generalized point to centroid distance p2c d by definition in addition we prove that if the membership grade matrix is nondegenerate any instance of the p2c d fits fcm directly finally extensive experiments have been conducted to demonstrate that the p2c d leads to the global convergence of fcm and that the clustering performances are significantly affected by the choices of distance functions

pattern identification in dynamical systems via symbolic time series analysis this paper presents symbolic time series analysis stsa of multi dimensional measurement data for pattern identification in dynamical systems the proposed methodology is built upon concepts derived from information theory and automata theory the objective is not merely to classify the time series patterns but also to identify the variations therein to achieve this goal a symbol alphabet is constructed from raw data through partitioning of the data space the maximum entropy method of partitioning is extended to multi dimensional space the resulting symbol sequences generated from time series data are used to model the dynamical information as finite state automata and the patterns are represented by the stationary state probability distributions a novel procedure for determining the structure of the finite state automata based on entropy rate is introduced the diversity among the observed patterns is quantified by a suitable measure the efficacy of the stsa technique for pattern identification is demonstrated via laboratory experimentation on nonlinear systems

dynamic pricing without knowing the demand function risk bounds and near optimal algorithms we consider a single product revenue management problem where given an initial inventory the objective is to dynamically adjust prices over a finite sales horizon to maximize expected revenues realized demand is observed over time but the underlying functional relationship between price and mean demand rate that governs these observations otherwise known as the demand function or demand curve is not known we consider two instances of this problem i a setting where the demand function is assumed to belong to a known parametric family with unknown parameter values and ii a setting where the demand function is assumed to belong to a broad class of functions that need not admit any parametric representation in each case we develop policies that learn the demand function on the fly and optimize prices based on that the performance of these algorithms is measured in terms of the regret the revenue loss relative to the maximal revenues that can be extracted when the demand function is known prior to the start of the selling season we derive lower bounds on the regret that hold for any admissible pricing policy and then show that our proposed algorithms achieve a regret that is close to this lower bound the magnitude of the regret can be interpreted as the economic value of prior knowledge on the demand function manifested as the revenue loss due to model uncertainty

a similarity based robust clustering method this paper presents an alternating optimization clustering procedure called a similarity based clustering method scm it is an effective and robust approach to clustering on the basis of a total similarity objective function related to the approximate density shape estimation we show that the data points in scm can self organize local optimal cluster number and volumes without using cluster validity functions or a variance covariance matrix the proposed clustering method is also robust to noise and outliers based on the influence function and gross error sensitivity analysis therefore scm exhibits three robust clustering characteristics 1 robust to the initialization cluster number and initial guesses 2 robust to cluster volumes ability to detect different volumes of clusters and 3 robust to noise and outliers several numerical data sets and actual data are used in the scm to show these good aspects the computational complexity of scm is also analyzed some experimental results of comparing the proposed scm with the existing methods show the superiority of the scm method

gossiping in minimal time the gossip problem involves communicating a unique item from each node in a graph to every other node this paper studies the minimum time required to do this under the weakest model of parallel communication which allows each node to participate in just one communication at a time as either sender or receiver a number of topologies are studied including the omplete graph grids hypercubes and rings definitive new optimal time algorithms are derived for complete graphs rings regular grids and toroidal grids that significantly extend existing results in particular an open problem about minimum time gossiping in complete graphs is settled specifically for a graph with n nodes at least log rho n communication steps where the logarithm is in the base of the golden ratio rho are required by any algorithm under the weakest model of communication this bound which is approximately 1 44 log 2 n can be realized for some networks and so the result is optimal

a cellular coevolutionary algorithm for image segmentation clustering is inherently a difficult problem both with respect to the definition of adequate models as well as to the optimization of the models we present a model for the cluster problem that does not need knowledge about the number of clusters a priori this property is among others useful in the image segmentation domain which we especially address further we propose a cellular coevolutionary algorithm for the optimization of the model within this scheme multiple agents are placed in a regular two dimensional 2 d grid representing the image which imposes neighboring relations on them the agents cooperatively consider pixel migration from one agent to the other in order to improve the homogeneity of the ensemble of the image regions they represent if the union of the regions of neighboring agents is homogeneous then the agents form alliances on the other hand if an agent discovers a deviant subject it isolates the subject in the experiments we show the effectiveness of the proposed method and compare it to other segmentation algorithms the efficiency can easily be improved by exploiting the intrinsic parallelism of the proposed method

wavelet based feature extraction using probabilistic finite state automata for pattern classification real time data driven pattern classification requires extraction of relevant features from the observed time series as low dimensional and yet information rich representations of the underlying dynamics these low dimensional features facilitate in situ decision making in diverse applications such as computer vision structural health monitoring and robotics wavelet transforms of time series have been widely used for feature extraction owing to their time frequency localization properties in this regard this paper presents a symbolic dynamics based method to model surface images generated by wavelet coefficients in the scale shift space these symbolic dynamics based models e g probabilistic finite state automata pfsa capture the relevant information embedded in the sensor data from the associated perron frobenius operators i e the state transition probability matrices the proposed method of pattern classification has been experimentally validated on laboratory apparatuses for two different applications i early detection of evolving damage in polycrystalline alloy structures and ii classification of mobile robots and their motion profiles

optimal algorithms for broadcast and gossip in the edge disjoint path modes the communication power of the one way and two way edge disjoint path modes for broadcast and gossip is investigated the complexity of communication algorithms is measured by the number of communication steps rounds the main results achieved are the following 1 for each connected graphgnofnnodes the complexity of broadcast ingn bmin gn satisfies log2n bmin gn log2n 1 the complete binary trees meet the upper bound and all graphs containing a hamiltonian path meet the lower bound 2 for each connected graphgnofnnodes the one way two way gossip complexityr gn r2 gn satisfies log2n r2 gn 2 log2n 1 1 44 log2n r gn 2 log2n 2 all these lower and upper bounds are shown to be sharp up to 1 3 all planar graphs ofnnodes and degreehhave a two way gossip complexity of at least 1 5log2n log2log2n 0 5log2h 8 and the two dimensional grid ofnnodes has the gossip complexity 1 5log2n log2log2n o 1 i e two dimensional grids are optimal gossip structures among planar graphs of bounded degree some upper bounds are also obtained for the one way mode 4 thed dimensional grid d 3 ofnnodes has the two way gossip complexity 1 1 d log2n log2nlog2n o d

faster and more robust point symmetry based k means algorithm based on the recently published point symmetry distance psd measure this paper presents a novel psd measure namely symmetry similarity level ssl operator for k means algorithm our proposed modified point symmetry based k means mpsk algorithm is more robust than the previous psk algorithm by su and chou not only the proposed mpsk algorithm is suitable for the symmetrical intra clusters as the psk algorithm does the proposed mpsk algorithm is also suitable for the symmetrical inter clusters in addition two speedup strategies are presented to reduce the time required in the proposed mpsk algorithm experimental results demonstrate the significant execution time improvement and the extension to the symmetrical inter clusters of the proposed mpsk algorithm when compared to the previous psk algorithm

regret and convergence bounds for a class of continuum armed bandit problems we consider a class of multi armed bandit problems where the set of available actions can be mapped to a convex compact region of ropfd sometimes denoted as the ldquocontinuum armed banditrdquo problem the paper establishes bounds on the efficiency of any arm selection procedure under certain conditions on the class of possible underlying reward functions both finite time lower bounds on the growth rate of the regret as well as asymptotic upper bounds on the rates of convergence of the selected control values to the optimum are derived we explicitly characterize the dependence of these convergence rates on the minimal rate of variation of the mean reward function in a neighborhood of the optimal control the bounds can be used to demonstrate the asymptotic optimality of the kiefer wolfowitz method of stochastic approximation with regard to a large class of possible mean reward functions

a robust automatic merging possibilistic clustering method krishnapuram and keller ieee trans fuzzy syst vol 1 no 2 pp 98 110 may 1993 proposed a possibilistic c means pcm clustering by relaxing the constraint in fuzzy c means fcms that the memberships of a data point across classes sum to 1 the pcm algorithm has a tendency to produce coincident clusters this can be a merit of pcm as a good mode seeking algorithm if initials and parameters are suitably chosen however the performance of pcm heavily depends on initializations and parameter selection in this paper we propose a mechanism of robust automatic merging we then create an automatic merging possibilistic clustering method am pcm where the proposed algorithm does not only solve these parameter selection and initialization problems but also obtains an optimal cluster number the proposed am pcm algorithm first uses all data points as initial cluster centers and then automatically merges these surrounding points around each cluster mode such that it can self organize data groups according to the original data structure the am pcm can exhibit the robustness to parameter noise cluster number different volumes and initializations the computational complexity of am pcm is also analyzed comparisons between am pcm and other clustering methods are made some numerical data and real datasets are used to show these good aspects of am pcm experimental results and comparisons actually demonstrate that the proposed am pcm is an effective and parameter free robust clustering algorithm

fuzzy c ordered medoids clustering for interval valued data fuzzy clustering for interval valued data helps us to find natural vague boundaries in such data the fuzzy c medoids clustering fcmdc method is one of the most popular clustering methods based on a partitioning around medoids approach however one of the greatest disadvantages of this method is its sensitivity to the presence of outliers in data this paper introduces a new robust fuzzy clustering method named fuzzy c ordered medoids clustering for interval valued data fcomdc id the huber s m estimators and the yager s ordered weighted averaging owa operators are used in the method proposed to make it robust to outliers the described algorithm is compared with the fuzzy c medoids method in the experiments performed on synthetic data with different types of outliers a real application of the fcomdc id is also provided highlightsfuzzy clustering for interval valued data helps us to find natural vague boundaries in such data a new robust fuzzy clustering method named fuzzy c ordered medoids clustering for interval valued data fcomdc id is proposedthe method uses both the huber s m estimators and the yager s owa operators to obtain its robustness experiments performed on synthetic data with different types of outliers and a real application are provided

iterative shrinking method for clustering problems agglomerative clustering generates the partition hierarchically by a sequence of merge operations we propose an alternative to the merge based approach by removing the clusters iteratively one by one until the desired number of clusters is reached we apply local optimization strategy by always removing the cluster that increases the distortion the least data structures and their update strategies are considered the proposed algorithm is applied as a crossover method in a genetic algorithm and compared against the best existing clustering algorithms the proposed method provides best performance in terms of minimizing intra cluster variance

the nearest subclass classifier a compromise between the nearest mean and nearest neighbor classifier we present the nearest subclass classifier nsc which is a classification algorithm that unifies the flexibility of the nearest neighbor classifier with the robustness of the nearest mean classifier the algorithm is based on the maximum variance cluster algorithm and as such it belongs to the class of prototype based classifiers the variance constraint parameter of the cluster algorithm serves to regularize the classifier that is to prevent overfitting with a low variance constraint value the classifier turns into the nearest neighbor classifier and with a high variance parameter it becomes the nearest mean classifier with the respective properties in other words the number of prototypes ranges from the whole training set to only one per class in the experiments we compared the nsc with regard to its performance and data set compression ratio to several other prototype based methods on several data sets the nsc performed similarly to the k nearest neighbor classifier which is a well established classifier in many domains also concerning storage requirements and classification speed the nsc has favorable properties so it gives a good compromise between classification performance and efficiency

on the complexity of bandwidth allocation in radio networks we define and study an optimization problem that is motivated by bandwidth allocation in radio networks because radio transmissions are subject to interference constraints in radio networks physical space is a common resource that the nodes have to share in such a way that concurrent transmissions do not interfere the bandwidth allocation problem we study under these constraints is the following given bandwidth traffic demands between the nodes of the network the objective is to schedule the radio transmissions in such a way that the traffic demands are satisfied the problem is similar to a multicommodity flow problem where the capacity constraints are replaced by the more complex notion of non interfering transmissions we provide a formal specification of the problem that we call round weighting by modeling non interfering radio transmissions as independent sets we relate the complexity of round weighting to the complexity of various independent set problems e g maximum weight independent set vertex coloring fractional coloring from this relation we deduce that in general round weighting is hard to approximate within n 1 e n being the size of the radio network we also provide polynomial exact or approximation algorithms e g in the following two cases a when the interference constraints are specific for instance for a network whose vertices belong to the euclidean space or b when the traffic demands are directed towards a unique node in the network also called gathering analogous to single commodity flow

minimum spanning tree based split and merge a hierarchical clustering method most clustering algorithms become ineffective when provided with unsuitable parameters or applied to datasets which are composed of clusters with diverse shapes sizes and densities to alleviate these deficiencies we propose a novel split and merge hierarchical clustering method in which a minimum spanning tree mst and an mst based graph are employed to guide the splitting and merging process in the splitting process vertices with high degrees in the mst based graph are selected as initial prototypes and k means is used to split the dataset in the merging process subgroup pairs are filtered and only neighboring pairs are considered for merge the proposed method requires no parameter except the number of clusters experimental results demonstrate its effectiveness both on synthetic and real datasets

genetic clustering for automatic evolution of clusters and application to image classification abstract in this article the searching capability of genetic algorithms has been exploited for automatically evolving the number of clusters as well as proper clustering of any data set a new string representation comprising both real numbers and the do not care symbol is used in order to encode a variable number of clusters the davies bouldin index is used as a measure of the validity of the clusters effectiveness of the genetic clustering scheme is demonstrated for both artificial and real life data sets utility of the genetic clustering technique is also demonstrated for a satellite image of a part of the city calcutta the proposed technique is able to distinguish some characteristic landcover types in the image

l 1 norm based fuzzy clustering abstract the paper presents the l1 version of the well known fuzzy clustering method namely fuzzy isodata proposed by bezdek and dunn due to their robustness l1 norm based methods gained much attention in statistics the presented fuzzy clustering problem uses the distance between observations and location parameter vectors which is based on the l1 norm instead of the inner product induced norm used in classical fuzzy isodata two alternative methods to solve the l1 fuzzy clustering problem are derived in practice both membership grades and location parameter vectors are unknown the paper presents two iterative algorithms each the implementation of the derived method finally numerical examples are presented one of them refers to famous iris data

optimality test for generalized fcm and its application to parameter selection in cluster analysis the fuzzy c means fcm clustering algorithm is the best known and most widely used method it was proven to converge to either a local minimum or saddle points by bezdek et al wei and mendel produced efficient optimality tests for fcm fixed points recently a weighting exponent selection for fcm was proposed by yu et al inspired by these results we unify several alternative fcm algorithms into one model called the generalized fuzzy c means gfcm this gfcm model presents a wide variation of fcm algorithms and can easily lead to new and interesting clustering algorithms moreover we construct a general optimality test for gfcm fixed points this is applied to theoretically choose the parameters in the gfcm model the experimental results demonstrate the precision of the theoretical analysis