scaling up cosine interesting pattern discovery a depth first method this paper presents an efficient algorithm called cosminer t for interesting pattern discovery the widely used cosine similarity found to possess the null invariance property and the anti cross support pattern property is adopted as the interestingness measure in cosminer t cosminer t is generally an fp growth like depth first traversal algorithm that rests on an important property of the cosine similarity the conditional anti monotone property camp the combined use of camp and the depth first support ascending traversal strategy enables the pre pruning of uninteresting patterns during the mining process of cosminer t extensive experiments demonstrate the high efficiency of cosminer t in interesting pattern discovery in comparison to the breath first strategy and the post evaluation strategy in particular cosminer t shows its capability in suppressing the generation of cross support patterns and discovering rare but truly interesting patterns finally an interesting case of landmark recognition is presented to illustrate the value of cosine interesting patterns found by cosminer t in real world applications

shape retrieval using hierarchical total bregman soft clustering in this paper we consider the family of total bregman divergences tbds as an efficient and robust distance measure to quantify the dissimilarity between shapes we use the tbd based l1 norm center as the representative of a set of shapes and call it the t center first we briefly present and analyze the properties of the tbds and t centers following our previous work in 1 then we prove that for any tbd there exists a distribution which belongs to the lifted exponential family lef of statistical distributions further we show that finding the maximum a posteriori map estimate of the parameters of the lifted exponential family distribution is equivalent to minimizing the tbd to find the t centers this leads to a new clustering technique namely the total bregman soft clustering algorithm we evaluate the tbd t center and the soft clustering algorithm on shape retrieval applications our shape retrieval framework is composed of three steps 1 extraction of the shape boundary points 2 affine alignment of the shapes and use of a gaussian mixture model gmm 2 3 4 to represent the aligned boundaries and 3 comparison of the gmms using tbd to find the best matches given a query shape to further speed up the shape retrieval algorithm we perform hierarchical clustering of the shapes using our total bregman soft clustering algorithm this enables us to compare the query with a small subset of shapes which are chosen to be the cluster t centers we evaluate our method on various public domain 2d and 3d databases and demonstrate comparable or better results than state of the art retrieval techniques

coordination of cluster ensembles via exact methods we present a novel optimization based method for the combination of cluster ensembles for the class of problems with intracluster criteria such as minimum sum of squares clustering mssc we propose a simple and efficient algorithm called examce for this class of problems that is inspired from a set partitioning formulation of the original clustering problem we prove some theoretical properties of the solutions produced by our algorithm and in particular that under general assumptions though the algorithm recombines solution fragments so as to find the solution of a set covering relaxation of the original formulation it is guaranteed to find better solutions than the ones in the ensemble for the mssc problem in particular a prototype implementation of our algorithm found a new better solution than the previously best known for 21 of the test instances of the 40 instance tsplib benchmark data sets used in and and found a worse quality solution than the best known only five times for other published benchmark data sets where the optimal mssc solution is known we match them the algorithm is particularly effective when the number of clusters is large in which case it is able to escape the local minima found by k means type algorithms by recombining the solutions in a set covering context we also establish the stability of the algorithm with extensive computational experiments by showing that multiple runs of examce for the same clustering problem instance produce high quality solutions whose adjusted rand index is consistently above 0 95 finally in experiments utilizing external criteria to compute the validity of clustering examce is capable of producing high quality results that are comparable in quality to those of the best known clustering algorithms

stratified feature sampling method for ensemble clustering of high dimensional data high dimensional data with thousands of features present a big challenge to current clustering algorithms sparsity noise and correlation of features are common characteristics of such data another common phenomenon is that clusters in such high dimensional data often exist in different subspaces ensemble clustering is emerging as a prominent technique for improving robustness stability and accuracy of high dimensional data clustering in this paper we propose a stratified sampling method for generating subspace component data sets in ensemble clustering of high dimensional data instead of randomly sampling a subset of features for each component data set in this method we first cluster the features of high dimensional data into a few feature groups called feature strata using stratified sampling we randomly sample some features from each feature stratum and merge the sampled features from different feature strata to generate a component data set in this way the component data sets have better representations of the clustering structure in the original data set comparing with random sampling and random projection methods in synthetic data analysis the component clustering by stratified sampling has demonstrated that the average clustering accuracy was increased without sacrificing clustering diversity we carried out a series of experiments on eight real world data sets from microarray text and image domains to evaluate ensemble clustering methods using three subspace component data generation methods and four consensus functions the experimental results consistently showed that the stratified sampling method produced the best ensemble clustering results in all data sets the ensemble clustering with stratified sampling also outperformed three other ensemble clustering methods which generate component clusters from the entire space of the original data highlightsa new component generation approach is proposed to produce ensemble components stratified sampling is used to generate subspace component data sets the component data can well represent the characteristics of the original data the proposed method can achieve a consistent performance in ensemble clustering the proposed method is easy to implement

a generalized kernel approach to dissimilarity based classification usually objects to be classified are represented by features in this paper we discuss an alternative object representation based on dissimilarity values if such distances separate the classes well the nearest neighbor method offers a good solution however dissimilarities used in practice are usually far from ideal and the performance of the nearest neighbor rule suffers from its sensitivity to noisy examples we show that other more global classification techniques are preferable to the nearest neighbor rule in such cases for classification purposes two different ways of using generalized dissimilarity kernels are considered in the first one distances are isometrically embedded in a pseudo euclidean space and the classification task is performed there in the second approach classifiers are built directly on distance kernels both approaches are described theoretically and then compared using experiments with different dissimilarity measures and datasets including degraded data simulating the problem of missing values

refined spectral clustering via embedded label propagation spectral clustering is a key research topic in the field of machine learning and data mining most of the existing spectral clustering algorithms are built on gaussian laplacian matrices which is sensitive to parameters we propose a novel parameter free distance consistent locally linear embedding the proposed distance consistent lle can promise that edges between closer data points are heavier we also propose a novel improved spectral clustering via embedded label propagation our algorithm is built on two advancements of the state of the art first is label propagation which propagates a node s labels to neighboring nodes according to their proximity we perform standard spectral clustering on original data and assign each cluster with nearest data points and then we propagate labels through dense unlabeled data regions second is manifold learning which has been widely used for its capacity to leverage the manifold structure of data points extensive experiments on various data sets validate the

extending information theoretic validity indices for fuzzy clustering previously eight popular information theoretic based cluster validity indices have been generalized and tested for probabilistic partitions built by the expectation maximization em algorithm for the gaussian mixture model however the analysis was limited to probabilistic clusters and there were limited explanations for differences in the performance of the indices in this paper we extend the tests to partitions found by fuzzy c means fcm and provide further explanations and insights about the performance of these indices of the eight generalized indices we advocate a normalized version of the soft mutual information cluster validity index nmi sm as the best overall choice as it outperforms the other seven indices for both fcm and em according to our tests on synthetic and real data the superiority of nmism is most pronounced for datasets with overlapped and or varying sized clusters finally we provide a theoretical analysis which helps explain the superior performance of nmism compared with the other three normalizations of soft mutual information

a novel clustering method based on hybrid k nearest neighbor graph abstract most of the existing clustering methods have difficulty in processing complex nonlinear data sets to remedy this deficiency in this paper a novel data model termed hybrid k nearest neighbor hknn graph which combines the advantages of mutual k nearest neighbor graph and k nearest neighbor graph is proposed to represent the nonlinear data sets moreover a clustering method based on the hknn graph chknn is proposed the chknn first generates several tight and small subclusters then merges these subclusters by exploiting the connectivity among them in order to select the optimal parameters for chknn we further propose an internal validity index termed k nearest neighbor index knni which can also be used to evaluate the validity of nonlinear clustering results by varying a control parameter experimental results on synthetic and real world data sets as well as that on the video clustering have demonstrated the significant improvement on performance over existing nonlinear clustering methods and internal validity indices

multiple view semi supervised dimensionality reduction multiple view data together with some domain knowledge in the form of pairwise constraints arise in various data mining applications how to learn a hidden consensus pattern in the low dimensional space is a challenging problem in this paper we propose a new method for multiple view semi supervised dimensionality reduction the pairwise constraints are used to derive embedding in each view and simultaneously the linear transformation is introduced to make different embeddings from different pattern spaces comparable hence the consensus pattern can be learned from multiple embeddings of multiple representations we derive an iterating algorithm to solve the above problem some theoretical analyses and out of sample extensions are also provided promising experiments on various data sets together with some important discussions are also presented to demonstrate the effectiveness of the proposed algorithm

computing the local consensus of trees the inference of consensus from a set of evolutionary trees is a fundamental problem in a number of fields such as biology and historical linguistics and many models for inferring this consensus have been proposed in this paper we present a model for deriving what we call a local consensus tree t from a set of trees cal t the model we propose presumes a function f called a total local consensus function which determines for every triple a of species the form that the local consensus tree should take on a we show that all local consensus trees when they exist can be constructed in polynomial time and that many fundamental problems can be solved in linear time we also consider partial local consensus functions and study optimization problems under this model we present linear time algorithms for several variations finally we point out that the local consensus approach ties together many previous approaches to constructing consensus trees

qos constrained multiuser peer to peer amplify and forward relay beamforming a wireless communication scenario is considered with k single antenna source destination pairs communicating through several half duplex amplify and forward mimo relays where each source is targeting only one destination the relay beamforming matrices are designed in order to minimize the total power transmitted from the relays subject to quality of service constraints on the received signal to interference plus noise ratio at each destination node due to the nonconvexity of this problem several approximations have been used in the literature to find a computationally efficient solution a novel solution technique is developed in which the problem is decomposed into a group of second order cone programs socps parameterized by k phase angles each associated with one of the constraints an iterative algorithm is proposed to search for the phase angles and the relay beamforming matrices sequentially however convergence to the global optimal beamforming matrices cannot be guaranteed two methods for searching for the optimal values of the phase angles are proposed from which the optimal beamforming matrices can be obtained using grid search and bisection and the convergence of these methods to the global optimal solution of the problem is proved numerical simulations are presented showing the superior performance of the proposed algorithms compared to earlier suboptimal approximations at the expense of a moderate increase in the computational complexity

information theoretic measures for clusterings comparison variants properties normalization and correction for chance information theoretic measures form a fundamental class of measures for comparing clusterings and have recently received increasing interest nevertheless a number of questions concerning their properties and inter relationships remain unresolved in this paper we perform an organized study of information theoretic measures for clustering comparison including several existing popular measures in the literature as well as some newly proposed ones we discuss and prove their important properties such as the metric property and the normalization property we then highlight to the clustering community the importance of correcting information theoretic measures for chance especially when the data size is small compared to the number of clusters present therein of the available information theoretic based measures we advocate the normalized information distance nid as a general measure of choice for it possesses concurrently several important properties such as being both a metric and a normalized measure admitting an exact analytical adjusted for chance form and using the nominal 0 1 range better than other normalized variants

combining multiple clusterings using similarity graph multiple clusterings are produced for various needs and reasons in both distributed and local environments combining multiple clusterings into a final clustering which has better overall quality has gained importance recently it is also expected that the final clustering is novel robust and scalable in order to solve this challenging problem we introduce a new graph based method our method uses the evidence accumulated in the previously obtained clusterings and produces a very good quality final clustering the number of clusters in the final clustering is obtained automatically this is another important advantage of our technique experimental test results on real and synthetically generated data sets demonstrate the effectiveness of our new method

a parameter free community detection method based on centrality and dispersion of nodes in complex networks abstract k means is a simple and efficient clustering algorithm to detect communities in networks however it may suffer from a bad choice of initial seeds also called centers that seriously affect the clustering accuracy and the convergence rate additionally in k means the number of communities should be specified in advance till now it is still an open problem on how to select initial seeds and how to determine the number of communities in this study a new parameter free community detection method named k rank d was proposed first based on the fact that good initial seeds usually have high importance and are dispersedly located in a network we proposed a modified pagerank centrality to evaluate the importance of a node and drew a decision graph to depict the importance and the dispersion of nodes then the initial seeds and the number of communities were selected from the decision graph actively and intuitively as the start parameter of k means experimental results on synthetic and real world networks demonstrate the superior performance of our approach over competing methods for community detection

joint image clustering and labeling by matrix factorization we propose a novel algorithm to cluster and annotate a set of input images jointly where the images are clustered into several discriminative groups and each group is identified with representative labels automatically for these purposes each input image is first represented by a distribution of candidate labels based on its similarity to images in a labeled reference image database a set of these label based representations are then refined collectively through a non negative matrix factorization with sparsity and orthogonality constraints the refined representations are employed to cluster and annotate the input images jointly the proposed approach demonstrates performance improvements in image clustering over existing techniques and illustrates competitive image labeling accuracy in both quantitative and qualitative evaluation in addition we extend our joint clustering and labeling framework to solving the weakly supervised image classification problem and obtain promising results

fast affinity propagation clustering a multilevel approach in this paper we propose a novel fast affinity propagation clustering approach fap fap simultaneously considers both local and global structure information contained in datasets and is a high quality multilevel graph partitioning method that can implement both vector based and graph based clustering first a new fast sampling algorithm fs is proposed to coarsen the input sparse graph and choose a small number of final representative exemplars then a density weighted spectral clustering method is presented to partition those exemplars on the global underlying structure of data manifold finally the cluster assignments of all data points can be achieved through their corresponding representative exemplars experimental results on two synthetic datasets and many real world datasets show that our algorithm outperforms the state of the art original affinity propagation and spectral clustering algorithms in terms of speed memory usage and quality on both vector based and graph based clustering

community mining with new node similarity by incorporating both global and local topological knowledge in a constrained random walk abstract detection of community is a crucial step to understand the structure and dynamics of complex networks most of conventional community detection methods focus on optimizing a certain objective function or on clustering nodes based on their similarities which leads to a phenomenon that they have preference for specific types of networks but are not general using constrained random walk we exploit global and local topology structures of network to propose a modified transition matrix and further to define a new similarity metric named isim between two nodes in contrast to the existing similarities isim does not work directly on the observed data but in a convergent stable space this feature makes isim robust to the observed noisy data in real world networks isim not only measures node s distance but also captures node s topology structure in network experiments on synthetic and real world networks demonstrate that isim can be successfully applied to community detection in broader types of networks and outperforms other community detection methods

construction of the similarity matrix for the spectral clustering method numerical experiments abstract spectral clustering is a powerful method for finding structure in a dataset through the eigenvectors of a similarity matrix it often outperforms traditional clustering algorithms such as k means when the structure of the individual clusters is highly non convex its accuracy depends on how the similarity between pairs of data points is defined two important items contribute to the construction of the similarity matrix the sparsity of the underlying weighted graph which depends mainly on the distances among data points and the similarity function when a gaussian similarity function is used the choice of the scale parameter can be critical in this paper we examine both items the sparsity and the selection of suitable s based either directly on the graph associated to the dataset or on the minimal spanning tree mst of the graph an extensive numerical experimentation on artificial and real world datasets has been carried out to compare the performances of the methods

bagging constraint score for feature selection with pairwise constraints constraint score is a recently proposed method for feature selection by using pairwise constraints which specify whether a pair of instances belongs to the same class or not it has been shown that the constraint score with only a small amount of pairwise constraints achieves comparable performance to those fully supervised feature selection methods such as fisher score however one major disadvantage of the constraint score is that its performance is dependent on a good selection on the composition and cardinality of constraint set which is very challenging in practice in this work we address the problem by importing bagging into constraint score and a new method called bagging constraint score bcs is proposed instead of seeking one appropriate constraint set for single constraint score in bcs we perform multiple constraint score each of which uses a bootstrapped subset of original given constraint set diversity analysis on individuals of ensemble shows that resampling pairwise constraints is helpful for simultaneously improving accuracy and diversity of individuals we conduct extensive experiments on a series of high dimensional datasets from uci repository and gene databases and the experimental results validate the effectiveness of the proposed method

discriminative metric learning for multi view graph partitioning we propose discriminative metric learning for multi view graph partitioning we envision the multi view graph as an adaptive dynamic system the intra view connections and the inter view couplings are interplayed extensive experiments have been conducted to show the effectiveness in recent years multi view graph partitioning has attracted more and more attention but most efforts have been made to develop graph partitioning approaches directly in the original topological structure in many real world applications graph may contain noisy links and the distance metric may not be so discriminative for revealing cluster structure this paper addresses the problem of discriminative metric learning for multi view graph partitioning in particular we propose a novel method called multi view dml abbr of multi view discriminative metric learning to transform the metric space in the original graph into a more discriminative metric space in which better graph partitioning results will be obtained we envision the multi view graph as an adaptive dynamic system where both the intra view connections and the inter view couplings are interplayed to gradually update the relation metric among nodes on the one hand the inter view coupling will be influenced by the intra view connection between two nodes on the other hand the inter view coupling will also affect the intra view connection such interplay eventually makes the whole graph reach a steady state which has a stronger cluster structure than the original graph extensive experiments are conducted on both synthetic and real world graphs to confirm that the proposed method is able to learn more discriminative metric

learning choquet integral based metrics for semisupervised clustering we consider an application of fuzzy measures to the problem of metric learning in semisupervised clustering we investigate the necessary and sufficient conditions on the underlying fuzzy measure that make the discrete choquet integral suitable for defining a metric as a byproduct we can obtain the analogous conditions for the ordered weighted averaging owa operators which constitute a special case we then generalize these results for power based choquet and owa operators we show that this metric learning problem can be formulated as a linear programming problem and specify the required sets of linear constraints we present the results of numerical experiments on artificial and real world datasets which illustrate the potential usefulness and limitations of this construction

stability analysis and control of a class of lpv systems with piecewise constant parameters stability criteria characterizing the asymptotic stability of a class of lpv systems with piecewise constant parameters under constant and minimum dwell time are derived it is shown that for such systems the conditions for the stability under minimum dwell time can be seen as a unifying stability concept lying in between quadratic and robust stability thereby including them as extremal cases the results are then extended to address the stabilization problem using a particular class of time dependent gain scheduled state feedback controllers several examples are given for illustration

sce a manifold regularized set covering method for data partitioning cluster analysis plays a very important role in data analysis in these years cluster ensemble as a cluster analysis tool has drawn much attention for its robustness stability and accuracy many efforts have been done to combine different initial clustering results into a single clustering solution with better performance however they neglect the structure information of the raw data in performing the cluster ensemble in this paper we propose a structural cluster ensemble sce algorithm for data partitioning formulated as a set covering problem in particular we construct a laplacian regularized objective function to capture the structure information among clusters moreover considering the importance of the discriminative information underlying in the initial clustering results we add a discriminative constraint into our proposed objective function finally we verify the performance of the sce algorithm on both synthetic and real data sets the experimental results show the effectiveness of our proposed method sce algorithm

from cluster ensemble to structure ensemble this paper investigates the problem of integrating multiple structures which are extracted from different sets of data points into a single unified structure we first propose a new generalized concept called structure ensemble for the fusion of multiple structures unlike traditional cluster ensemble approaches the main objective of which is to align individual labels obtained from different clustering solutions the structure ensemble approach focuses on how to unify the structures obtained from different data sources based on this framework a new structure ensemble approach called the probabilistic bagging based structure ensemble approach bsea is designed which integrates the bagging technique the force based self organizing map fbsom and the normalized cut algorithm into the proposed framework bsea views structures obtained from different datasets generated by the bagging technique as nodes in a graph and adopts graph theory to find the most representative structure in addition the force based self organizing map fbsom which is a generalized form of som is proposed to serve as the basic clustering algorithm in the structure ensemble framework finally a new external index called correlation index ci which considers the correlation relationship of both the similarity and dissimilarity between the predicted solution and the true solution is proposed to evaluate the performance of bsea the experiments show that i the performance of bsea outperforms most of the state of the art clustering approaches and ii bsea performs well on datasets from the uci repository and real cancer gene expression profiles

optimal clustering in the context of overlapping cluster analysis in this paper we give a general definition for the concept optimal clustering which is applicable to overlapping clusterings overlapping clusterings are a generalization of hard clusterings and their structure is formally developed in this paper it is generally assumed that the domain of clustering is too heuristic to develop a general i e axiomatic definition for an optimal clustering it is shown however that such a definition can be given within the domain of overlapping clusterings using the new concept of dual clustering developed in this paper a second concept that underlies our definition of optimal clustering is the average clustering also playing an important role in the domain of cluster ensembles using the general concepts discussed in this paper it is then shown that under some conditions it is assured that the final hard clustering extracted by majority vote from a given set of clusterings is optimal over all hard clusterings unlike traditional research related to validating clusterings we do not develop a new cluster validation measure on top of the many existing ones but rather we develop a general framework for cluster validation measures at least within the domain of overlapping clusterings this framework allows to develop some general theorems about clustering

centroid index cluster level similarity measure abstract in clustering algorithm one of the main challenges is to solve the global allocation of the clusters instead of just local tuning of the partition borders despite this all external cluster validity indexes calculate only point level differences of two partitions without any direct information about how similar their cluster level structures are in this paper we introduce a cluster level index called centroid index the measure is intuitive simple to implement fast to compute and applicable in case of model mismatch as well to a certain extent we expect it to generalize other clustering models beyond the centroid based k means as well

cluster ensembles a knowledge reuse framework for combining multiple partitions this paper introduces the problem of combining multiple partitionings of a set of objects into a single consolidated clustering without accessing the features or algorithms that determined these partitionings we first identify several application scenarios for the resultant knowledge reuse framework that we call cluster ensembles the cluster ensemble problem is then formalized as a combinatorial optimization problem in terms of shared mutual information in addition to a direct maximization approach we propose three effective and efficient techniques for obtaining high quality combiners consensus functions the first combiner induces a similarity measure from the partitionings and then reclusters the objects the second combiner is based on hypergraph partitioning the third one collapses groups of clusters into meta clusters which then compete for each object to determine the combined clustering due to the low computational costs of our techniques it is quite feasible to use a supra consensus function that evaluates all three approaches against the objective function and picks the best solution for a given situation we evaluate the effectiveness of cluster ensembles in three qualitatively different application scenarios i where the original clusters were formed based on non identical sets of features ii where the original clustering algorithms worked on non identical sets of objects and iii where a common data set is used and the main purpose of combining multiple clusterings is to improve the quality and robustness of the solution promising results are obtained in all three situations for synthetic as well as real data sets