kernel based hard clustering methods in the feature space with automatic variable weighting abstract this paper presents variable wise kernel hard clustering algorithms in the feature space in which dissimilarity measures are obtained as sums of squared distances between patterns and centroids computed individually for each variable by means of kernels the methods proposed in this paper are supported by the fact that a kernel function can be written as a sum of kernel functions evaluated on each variable separately the main advantage of this approach is that it allows the use of adaptive distances which are suitable to learn the weights of the variables on each cluster providing a better performance moreover various partition and cluster interpretation tools are introduced experiments with synthetic and benchmark datasets show the usefulness of the proposed algorithms and the merit of the partition and cluster interpretation tools

bounds for the zeros of polynomials from eigenvalues and singular values of some companion matrices abstract let a monic polynomial pn x xn a1xn 1 an aj plane1d 49e j 1 2 n be given bounds for the eigenvalues and the computation of the singular values of some special companion matrices give bounds for the zeros of the polynomial pn which improve classical and recently found ones also some new bounds for the zeros are given

statistical approach to normalization of feature vectors and clustering of mixed datasets normalization of feature vectors of datasets is widely used in a number of fields of data mining in particular in cluster analysis where it is used to prevent features with large numerical values

bisimulation on speed worst case efficiency this paper introduces a novel bi simulation based faster than preorder which relates asynchronous processes where the relative speeds of system components are indeterminate with respect to their worst case timing behavior the study is conducted for a conservative extension of the process algebra ccs called tacs which permits the specification of upper time bounds on action occurrences tacs complements work in plain process algebras which compares asynchronous processes with respect to their functional reactive behavior only and in timed process algebras which focus on analyzing synchronous processes the most unusual contribution is in showing that the proposed faster than preorder coincides with several other preorders two of which consider the absolute times at which actions occur in system runs the paper also develops the semantic theory of tacs by studying congruence properties equational laws and abstractions from internal actions two examples one dealing with mail delivery and one relating two implementations of a simple storage system testify to the practical utility of the new theory

multiple kernel clustering based on centered kernel alignment multiple kernel clustering mkc which performs kernel based data fusion for data clustering is an emerging topic it aims at solving clustering problems with multiple cues most mkc methods usually extend existing clustering methods with a multiple kernel learning mkl setting in this paper we propose a novel mkc method that is different from those popular approaches centered kernel alignment an effective kernel evaluation measure is employed in order to unify the two tasks of clustering and mkl into a single optimization framework to solve the formulated optimization problem an efficient two step iterative algorithm is developed experiments on several uci datasets and face image datasets validate the effectiveness and efficiency of our mkc algorithm

a survey of kernel and spectral methods for clustering clustering algorithms are a useful tool to explore data structures and have been employed in many disciplines the focus of this paper is the partitioning clustering problem with a special interest in two recent approaches kernel and spectral methods the aim of this paper is to present a survey of kernel and spectral clustering methods two approaches able to produce nonlinear separating hypersurfaces between clusters the presented kernel clustering methods are the kernel version of many classical clustering algorithms e g k means som and neural gas spectral clustering arise from concepts in spectral graph theory and the clustering problem is configured as a graph cut problem where an appropriate objective function has to be optimized an explicit proof of the fact that these two paradigms have the same objective is reported since it has been proven that these two seemingly different approaches have the same mathematical foundation besides fuzzy kernel clustering methods are presented as extensions of kernel k means clustering algorithm

extremal sparsity of the companion matrix of a polynomial abstract let c be the companion matrix of a monic polynomial p over a field f we prove that if a is a matrix whose entries are rational functions of the coefficients of p over f and whose characteristic polynomial is p then a has at least as many nonzero entries as c

fast computation of the zeros of a polynomial via factorization of the companion matrix a new fast algorithm for computing the zeros of a polynomial in o n 2 time using o n memory is developed the eigenvalues of the frobenius companion matrix are computed by applying a nonunitary analogue of francis s implicitly shifted qr algorithm to a factored form of the matrix the algorithm achieves high speed and low memory use by preserving the factored form it also provides a residual and an error estimate for each root numerical tests confirm the high speed of the algorithm

robust quantization for general similarity search the recent years have witnessed the emerging of vector quantization vq techniques for efficient similarity search vq partitions the feature space into a set of codewords and encodes data points as integer indices using the codewords then the distance between data points can be efficiently approximated by simple memory lookup operations by the compact quantization the storage cost and searching complexity are significantly reduced thereby facilitating efficient large scale similarity search however the performance of several celebrated vq approaches degrades significantly when dealing with noisy data in addition it can barely facilitate a wide range of applications as the distortion measurement only limits to ell 2 norm to address the shortcomings of the squared euclidean ell 2 2 norm loss function employed by the vq approaches in this paper we propose a novel robust and general vq framework named rgvq to enhance both robustness and generalization of vq approaches specifically a ell p q norm loss function is proposed to conduct the ell p norm similarity search rather than the ell 2 norm search and the q th order loss is used to enhance the robustness despite the fact that changing the loss function to ell p q norm makes vq approaches more robust and generic it brings us a challenge that a non smooth and non convex orthogonality constrained ell p q norm function has to be minimized to solve this problem we propose a novel and efficient optimization scheme and specify it to vq approaches and theoretically prove its convergence extensive experiments on benchmark data sets demonstrate that the proposed rgvq is better than the original vq for several approaches especially when searching similarity in noisy data

worst case and smoothed analysis of the icp algorithm with an application to the k means method we show a worst case lower bound and a smoothed upper bound on the number of iterations performed by the iterative closest point icp algorithm first proposed by besl and mckay the algorithm is widely used in computational geometry where it is known for its simplicity and its observed speed the theoretical study of icp was initiated by ezra sharir and efrat who showed that the worst case running time to align two sets of n points in mathbb r d is between omega n log n and o n 2d d we substantially tighten this gap by improving the lower bound to omega n d d 1 to help reconcile this bound with the algorithm s observed speed we also show that the smoothed complexity of icp is polynomial independent of the dimensionality of the data using similar methods we improve the best known smoothed upper bound for the popular k means method to n o k once again independent of the dimension

minkowski metric feature weighting and anomalous cluster initializing in k means clustering this paper represents another step in overcoming a drawback of k means its lack of defense against noisy features using feature weights in the criterion the weighted k means method by huang et al 2008 2004 2005 5 7 is extended to the corresponding minkowski metric for measuring distances under minkowski metric the feature weights become intuitively appealing feature rescaling factors in a conventional k means criterion to see how this can be used in addressing another issue of k means the initial setting a method to initialize k means with anomalous clusters is adapted the minkowski metric based method is experimentally validated on datasets from the uci machine learning repository and generated sets of gaussian clusters both as they are and with additional uniform random noise features and appears to be competitive in comparison with other k means based feature weighting algorithms

a note on companion matrices abstract we show that the usual companion matrix of a polynomial of degree n can be factored into a product of n matrices n 1 of them being the identity matrix in which a 2 2 identity submatrix in two consecutive rows and columns is replaced by an appropriate 2 2 matrix the remaining being the identity matrix with the last entry replaced by possibly different entry by a certain similarity transformation we obtain a simple new companion matrix in a pentadiagonal form some generalizations are also possible

information theoretic clustering using a k nearest neighbors approach abstract we develop a new non parametric information theoretic clustering algorithm based on implicit estimation of cluster densities using the k nearest neighbors k nn approach compared to a kernel based procedure our hierarchical k nn approach is very robust with respect to the parameter choices with a key ability to detect clusters of vastly different scales of particular importance is the use of two different values of k depending on the evaluation of within cluster entropy or across cluster cross entropy and the use of an ensemble clustering approach wherein different clustering solutions vote in order to obtain the final clustering we conduct clustering experiments and report promising results

joint image denoising using adaptive principal component analysis and self similarity the non local means nlm has attracted enormous interest in image denoising problem in recent years in this paper we propose an efficient joint denoising algorithm based on adaptive principal component analysis pca and self similarity that improves the predictability of pixel intensities in reconstructed images the proposed algorithm consists of two successive steps without iteration the low rank approximation based on parallel analysis and the collaborative filtering first for a pixel and its nearest neighbors the training samples in a local search window are selected to form the similar patch group by the block matching method next it is factorized by singular value decomposition svd whose left and right orthogonal basis denote local and non local image features respectively the adaptive pca automatically chooses the local signal subspace dimensionality of the noisy similar patch group in the svd domain by the refined parallel analysis with monte carlo simulation thus image features can be well preserved after dimensionality reduction and simultaneously the noise is almost eliminated then after the inverse svd transform the denoised image is reconstructed from the aggregate filtered patches by the weighted average method finally the collaborative wiener filtering is used to further remove the noise the experimental results validate its generality and effectiveness in a wide range of the noisy images the proposed algorithm not only produces very promising denoising results that outperforms the state of the art methods in most cases but also adapts to a variety of noise levels

a complete axiomatization of timed bisimulation for a class of timed regular behaviours abstract one of the most satisfactory results in process theory is milner s axiomatization of strong bisimulation for regular ccs this result holds for open terms with finite state recursion wang has shown that timed bisimulation can also be axiomatized but only for closed terms without recursion in this paper we provide an axiomatization for timed bisimulation of open terms with finite state recursion

a process algebra for timed systems a standard process algebra is extended by a new action which is meant to denote idling until the next clock cycle a semantic theory based on testing is developed for the new language this is characterised in terms of barbs a variety of ready traces and also characterised as the initial theory generated by a set of equations

approximate nearest neighbor search for l p spaces 2 p via embeddings abstract while the problem of approximate nearest neighbor search has been well studied for euclidean space and l 1 few non trivial algorithms are known for l p when 2 p in this paper we revisit this fundamental problem and present approximate nearest neighbor search algorithms which give the best known approximation factor guarantees in this setting

distance metric learning for soft subspace clustering in composite kernel space soft subspace clustering algorithms have been successfully used for high dimensional data in recent years however the existing algorithms often utilize only one distance function to evaluate the distance between data items on each feature which cannot deal with datasets with complex inner structures in this paper a composite kernel space cks is constructed based on a set of basis kernels and a novel framework of soft subspace clustering is proposed by integrating distance metric learning in the cks two soft subspace clustering algorithms i e entropy weighting fuzzy clustering in cks for kernel space cks ewfc k and feature space cks ewfc f are thus developed in both algorithms the prototype in the feature space is mapped into the cks by multiple simultaneous mappings one mapping for each cluster which is distinct from existing kernel based clustering algorithms by evaluating the distance on each feature in the cks both cks ewfc k and cks ewfc f learn the distance function adaptively during the clustering process experimental results have demonstrated that the proposed algorithms in general outperform classical clustering algorithms and are immune to ineffective kernels and irrelevant features in soft subspace the composite kernel space is constructed based on a set of basis kernels the general form of soft subspace clustering in cks is presented cks ewfc k and cks ewfc f are proposed under the framework of cks ssc the properties of cks ewfc k and cks ewfc f are investigated both cks ewfc k and cks ewfc f are immune to ineffective kernels

is observational congruence on μ expressions axiomatisable in equational horn logic it is well known that bisimulation on m expressions cannot be finitely axiomatised in equational logic complete axiomatisations such as those of milner and bloom esik necessarily involve implicational rules however both systems rely on features beyond pure equational horn logic either rules that are impure by involving non equational side conditions or rules that are schematically infinitary like the congruence rule which is not horn it is an open question whether these complications cannot be avoided in the proof theoretically and computationally clean and powerful setting of second order equational horn logic this paper presents a positive and a negative result regarding the axiomatisability of observational congruence in equational horn logic firstly we show how milner s impure rule system can be reworked into a pure horn axiomatisation that is complete for guarded processes secondly we prove that for unguarded processes both milner s unique fixed point rule and bloom esik s ga rule are incomplete without the congruence rule and neither system has a complete extension in rank 1 equational axioms it remains open whether there are higher rank equational axioms or other horn rules which would render milner s or bloom esik s axiomatisations complete

the p norm push a simple convex ranking algorithm that concentrates at the top of the list we are interested in supervised ranking algorithms that perform especially well near the top of the ranked list and are only required to perform sufficiently well on the rest of the list in this work we provide a general form of convex objective that gives high scoring examples more importance this push near the top of the list can be chosen arbitrarily large or small based on the preference of the user we choose lp norms to provide a specific type of push if the user sets p larger the objective concentrates harder on the top of the list we derive a generalization bound based on the p norm objective working around the natural asymmetry of the problem we then derive a boosting style algorithm for the problem of ranking with a push at the top the usefulness of the algorithm is illustrated through experiments on repository data we prove that the minimizer of the algorithm s objective is unique in a specific sense furthermore we illustrate how our objective is related to quality measurements for information retrieval

spectrally arbitrary patterns abstract an n n sign pattern matrix s is an inertially arbitrary pattern iap if each nonnegative triple n 1 n 2 n 3 with n 1 n 2 n 3 n is the inertia of a matrix with sign pattern s analogously s is a spectrally arbitrary pattern sap if for any given real monic polynomial r x of order n there is a matrix with sign pattern s and characteristic polynomial r x focusing on tree sign patterns consider the n n tridiagonal sign pattern t n with each superdiagonal entry positive each subdiagonal entry negative the 1 1 entry negative the n n entry positive and every other entry zero it is conjectured that t n is an iap by constructing matrices a n with pattern t n it is proved that t n allows any inertia with n 3 0 1 2 n 1 n for all n 2 this leads to a proof of the conjecture for n 5 the truth of the conjecture is extended to n 7 by showing the stronger result that t n is a sap the proof of this latter statement involves finding a matrix a n with pattern t n that is nilpotent further questions about patterns that are saps and iaps are considered

companion matrix patterns abstract companion matrices especially the frobenius companion matrices are used in algorithms for finding roots of polynomials and are also used to find bounds on eigenvalues of matrices in 2003 fiedler introduced a larger class of companion matrices that includes the frobenius companion matrices one property of the combinatorial pattern of these companion matrices is that up to diagonal similarity they uniquely realize every possible spectrum of a real matrix we characterize matrix patterns that have this property and consequently introduce more companion matrix patterns we observe that each fiedler companion matrix is permutationally similar to a unit hessenberg matrix we provide digraph characterizations of the classes of patterns described and in particular all sparse companion matrices noting that there are companion matrices that are not sparse

the p norm generalization of the lms algorithm for adaptive filtering recently much work has been done analyzing online machine learning algorithms in a worst case setting where no probabilistic assumptions are made about the data this is analogous to the h sup spl infin setting used in adaptive linear filtering bregman divergences have become a standard tool for analyzing online machine learning algorithms using these divergences we motivate a generalization of the least mean squared lms algorithm the loss bounds for these so called p norm algorithms involve other norms than the standard 2 norm the bounds can be significantly better if a large proportion of the input variables are irrelevant i e if the weight vector we are trying to learn is sparse we also prove results for nonstationary targets we only know how to apply kernel methods to the standard lms algorithm i e p 2 however even in the general p norm case we can handle generalized linear models where the output of the system is a linear function combined with a nonlinear transfer function e g the logistic sigmoid

recovering the number of clusters in data sets with noise features using feature rescaling factors in this paper we introduce three methods for re scaling data sets aiming at improving the likelihood of clustering validity indexes to return the true number of spherical gaussian clusters with additional noise features our method obtains feature re scaling factors taking into account the structure of a given data set and the intuitive idea that different features may have different degrees of relevance at different clusters we experiment with the silhouette using squared euclidean manhattan and the pth power of the minkowski distance dunn s calinski harabasz and hartigan indexes on data sets with spherical gaussian clusters with and without noise features we conclude that our methods indeed increase the chances of estimating the true number of clusters in a data set

algebraic laws for nondeterminism and concurrency since a nondeterministic and concurrent program may in general communicate repeatedly with its environment its meaning cannot be presented naturally as an input output function as is often done in the denotational approach to semantics in this paper an alternative is put forth first a definition is given of what it is for two programs or program parts to be equivalent for all observers then two program parts are said to be observation congruent if they are in all program contexts equivalent the behavior of a program part that is its meaning is defined to be its observation congruence class the paper demonstrates for a sequence of simple languages expressing finite terminating behaviors that in each case observation congruence can be axiomatized algebraically moreover with the addition of recursion and another simple extension the algebraic language described here becomes a calculus for writing and specifying concurrent programs and for proving their properties

a complete axiomatisation for observational congruence of finite state behaviours abstract finite state automata with non determinism and silent transitions can be interpreted not as subsets of the free monoid as in classical automata theory but as congruence classes under a congruence relation based upon the notion of weak bisimulation or observational equivalence due to park and milner in this paper a complete axiomatisation for this congruence is presented it extends the previously known complete axiomatisation by hennessy and milner for the case when all computations are finite the extension consists of five simple rules for recursion

the minkowski central partition as a pointer to a suitable distance exponent and consensus partitioning we generate optimal minkowski partitions at various values of the exponent p we define the minkowski profile based on the average similarity between partitions minkowski profile is highly correlated with ari vectors related to the ground truth we define the central minkowski partition which can serve as a consensus partition the silhouette width should be used for selecting the optimal minkowski exponent the minkowski weighted k means mwk means is a recently developed clustering algorithm capable of computing feature weights the cluster specific weights in mwk means follow the intuitive idea that a feature with low variance should have a greater weight than a feature with high variance the final clustering found by this algorithm depends on the selection of the minkowski distance exponent this paper explores the possibility of using the central minkowski partition in the ensemble of all minkowski partitions for selecting an optimal value of the minkowski exponent the central minkowski partition appears to be also a good consensus partition furthermore we discovered some striking correlation results between the minkowski profile defined as a mapping of the minkowski exponent values into the average similarity values of the optimal minkowski partitions and the adjusted rand index vectors resulting from the comparison of the obtained partitions to the ground truth our findings were confirmed by a series of computational experiments involving synthetic gaussian clusters and real world data

condition numbers for inversion of fiedler companion matrices abstract the fiedler matrices of a monic polynomial p z of degree n are n n matrices with characteristic polynomial equal to p z and whose nonzero entries are either 1 or minus the coefficients of p z fiedler matrices include as particular cases the classical frobenius companion forms of p z frobenius companion matrices appear frequently in the literature on control and signal processing but it is well known that they posses many properties that are undesirable numerically which limit their use in applications in particular as n increases frobenius companion matrices are often nearly singular i e their condition numbers for inversion are very large therefore it is natural to investigate whether other fiedler matrices are better conditioned than the frobenius companion matrices or not in this paper we present explicit expressions for the condition numbers for inversion of all fiedler matrices with respect the frobenius norm i e a f ij a ij 2 this allows us to get a very simple criterion for ordering all fiedler matrices according to increasing condition numbers and to provide lower and upper bounds on the ratio of the condition numbers of any pair of fiedler matrices these results establish that if p 0 1 then the frobenius companion matrices have the largest condition number among all fiedler matrices of p z and that if p 0 1 then the frobenius companion matrices have the smallest condition number we also provide families of polynomials where the ratio of the condition numbers of pairs of fiedler matrices can be arbitrarily large and prove that this can only happen when both fiedler matrices are very ill conditioned we finally study some properties of the singular values of fiedler matrices and determine how many of the singular values of a fiedler matrix are equal to one

potentially nilpotent sign pattern matrices abstract a sign pattern is said to be potentially nilpotent if it allows nilpotence in this paper a number of qualitative necessary or sufficient conditions for a sign pattern to allow nilpotence are established the sign patterns that allow nilpotence of index 2 are investigated for orders up to three potentially nilpotent sign patterns are characterized potentially nilpotent tree sign patterns are also explored

an initialization method for the k means algorithm using neighborhood model as a simple clustering method the traditional k means algorithm has been widely discussed and applied in pattern recognition and machine learning however the k means algorithm could not guarantee unique clustering result because initial cluster centers are chosen randomly in this paper the cohesion degree of the neighborhood of an object and the coupling degree between neighborhoods of objects are defined based on the neighborhood based rough set model furthermore a new initialization method is proposed and the corresponding time complexity is analyzed as well we study the influence of the three norms on clustering and compare the clustering results of the k means with the three different initialization methods the experimental results illustrate the effectiveness of the proposed method

the continuing influence of fiedler s work on companion matrices this is a reconstruction in article like form of a talk given at the minisymposium in honor of miroslav fiedler at the 17th ilas conference held at tu braunschweig germany on thurs 25 aug 2011

kernel based hard clustering methods with kernelization of the metric and automatic weighting of the variables this paper presents kernel based hard clustering methods with kernelization of the metric and automatic weighting of the variables the proposed methodology is supported by the fact that a kernel function can be written as a sum of kernels evaluated separately on each variable thus in the proposed algorithms dissimilarity measures are obtained as sums of euclidean distances between patterns and centroids computed individually for each variable by means of kernels the main advantage of this approach over the conventional approach is that it allows the use of kernelized adaptive distances which are suitable to learn the weights of the variables dynamically improving the performance of the algorithms moreover various partition and cluster interpretation tools are introduced experiments with a number of benchmark datasets corroborate the usefulness of the proposed algorithms and the merit of the partition and cluster interpretation tools highlightspresents kernel based clustering methods with automatic weighting of the variables kernelized local and global adaptive distances are introduced the proposed algorithms are suitable to learn the weights of the variables partition and cluster interpretation tools are given experiments with several benchmark data sets corroborate the proposed methods

single point iterative weighted fuzzy c means clustering algorithm for remote sensing image segmentation in this paper a remote sensing image segmentation procedure that utilizes a single point iterative weighted fuzzy c means clustering algorithm is proposed based upon the prior information this method can solve the fuzzy c means algorithm s problem that the clustering quality is greatly affected by the data distributing and the stochastic initializing the centrals of clustering after the probability statistics of original data the weights of data attribute are designed to adjust original samples to the uniform distribution and added in the process of cyclic iteration which could be suitable for the character of fuzzy c means algorithm so as to improve the precision furthermore appropriate initial clustering centers adjacent to the actual final clustering centers can be found by the proposed single point adjustment method which could promote the convergence speed of the overall iterative process and drastically reduce the calculation time otherwise the modified algorithm is updated from multidimensional data analysis to color images clustering moreover with the comparison experiments of the uci data sets public berkeley segmentation dataset and the actual remote sensing data the real validity of proposed algorithm is proved

an optimization algorithm for clustering using weighted dissimilarity measures one of the main problems in cluster analysis is the weighting of attributes so as to discover structures that may be present by using weighted dissimilarity measures for objects a new approach is developed which allows the use of the k means type paradigm to efficiently cluster large data sets the optimization algorithm is presented and the effectiveness of the algorithm is demonstrated with both synthetic and real data sets

estimating the number of clusters in a data set via the gap statistic we propose a method the gap statistic for estimating the number of clusters groups in a set of data the technique uses the output of any clustering algorithm e g k means or hierarchical comparing the change in within cluster dispersion with that expected under an appropriate reference null distribution some theory is developed for the proposal and a simulation study shows that the gap statistic usually outperforms other methods that have been proposed in the literature