The analysis of a simple k-means clustering algorithm

*Center for Automat ion Research, University of Maryland College Park, Maryland. Email: kanungo©cfar.umd.edu. tDepar tment of Computer Science, University of Maryland, College Park, Maryland. Email: mount0cs.umd.edu. This research was part ial ly supported by the NSF under grant CCR-9712379. $Department of Mathematics and Computer Science, BarIlan University, Ramat-Gan 52900, Israel and Center for Automation Research, University of Maryland, College Park, Maryland 20742. Email: nathan@macs.biu, ac. £1. § The Johns Hopkins University Applied Physics Laboratory, Laurel, Maryland. Email: c h r i s t i n e . p i a t k o @ j h u a p l . e d u . ¶Depar tment of Computer Science, University of the Distr ict of Columbia, Washington, DC, and Center for Automation Research, University of Maryland, College Park, Maryland. Email: ruth@cfar.umd.edu. IIDepartment of Computer Science and Information Systems, The American University, Washington, DC. Email: awu@american, edu. Permission to make digital or hard copies of all or part of this work tbr personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a lbe. Computational Geometry 2000 Hong Kong China Copyright ACM 2000 1-58113-224-7/00/6...$5.00 1. I N T R O D U C T I O N Clustering problems arise in many different applications, such as da ta mining and knowledge discovery [16], da ta compression and vector quantization [21], and pat tern recognition and pat tern classification [13]. One impor tant class of clustering problems, sometimes called k-means clustering, is the following: Given a set of n da ta points in real ddimensional space, R d, and an integer k, determine a set of k points in R d, called centers, so as to minimize the mean squared distance from each da ta point to its nearest center. This measure is often called the squared-error distortion [24, 21], and this type of clustering falls into the general category of variance-based clustering [23, 22]. The k-means problem is closely related to a number of other clustering problems, such as the Euclidean k-medians problem [2, 26], in which the objective is to minimize the sum of distances, and the geometric k-center problem [1], in which the objective is to minimize the maximum distance. Also see Capolyeas et al. [9] and Jaln and Dubes [24] for other generalizations and issues in geometric clustering problems. An asymptotically efficient approximation for the k-means clustering problem has been presented by Matou~ek [31], but the very large constant factors involved with the construction suggest that it is not a good candidate for implementation. One of the most popular heuristics for solving the k-means problem is based on a simple i terative scheme for finding a locally minimal solution. This algorithm is often called the k-means algorithm [18, 28]. There are a number of variants of this algorithm, so to clarify which version we are using, we will refer to it as Lloyd's algorithm. (To be even more accurate, it would be bet ter to call it the generalized Lloyd's algorithm, since Lloyd's original result was for scalar da ta [27].) Here is how Lloyd's algorithm works. Given any set of k centers, for each center ci, let V/ denote the set of da ta points for which ci is the nearest neighbor. (Equivalently, V/ is the set of da ta points lyingAn the Voronoi cell of ci relative to the set of centers.) For the next i terat ion of the algorithm, replace ci with the centroid of I~ and update V/ accordingly. These two steps are repeated until some convergence conditions have been met. See Faber [15] for descriptions of other variants of this algorithm. For points in general position, the algorithm will eventually converge to a point tha t is a local minimum. This is true