Talk:K-means clustering

message: Some examples? --Abdull 16:32, 21 February 2006 (UTC)
thread_title: Untitled 1: Examples?

message: I assume that the comment in the end of the article ("Also, the algorithm works well only when spherical clusters are naturally available in data.") is very much dependent on the distance metric used... --Andkaha(talk) 15:52, 20 July 2006 (UTC)
thread_title:  Spherical clusters 

message: Suggestion: Make title to lowercase (I believe k-means is the consensus and not K-means) --Chrike 03:04, 19 January 2007 (UTC)
thread_title: Untitled 2: Lowercase

message: Originally I started working on replacement images for the examples because they weren't licensed properly and I figured replacement was easier than tracking down the owner.  However, they have sinced been properly licensed but I went ahead and replaced them because the new images don't show external software (which I beleive detracts from the quality of the image) and are in .svg format.  However, I could be mistaken and do not wish to unfairly promote my own work, so if someone preferred the old images they can revert. -192.25.240.225 19:30, 26 July 2007 (UTC)
thread_title:  Image Replacement 

message: Why does the article put so much emphasis on the K-Means++ algorithm? If this algorithm really was so awesome as the text suggest then I would expect the "k-means++ The Advantages of Careful Seeding" article to be often quoted by scientific articles. It is not, and I think the reason is quite simple. It is an obscure article that "prove" its results using very specialized datasets that does not match what K-Means is used for out in the wild. I seriously doubt the results will hold if K-means++ is tried in more general situations.  —Preceding unsigned comment added by 81.216.185.173 (talk) 10:32, 29 November 2008 (UTC)
thread_title: Untitled 3: Emphasis on K-Means++

message: I propose changing the title to K-means clustering for two reasons:
* It explains what it does (it clusters around k means)
* There appear to be a number of algorithms, so referring to it as "K-means algorithm" is a misnomer.
comments? -3mta3 (talk) 09:11, 7 April 2009 (UTC)
replies: text: http://www.cc.gatech.edu/~vempala/papers/dfkvv.pdf (NP-hard even if k=2), http://www.imsc.res.in/~meena/papers/kmeans.pdf (NP-hard even if d=2). I do not think the second paper has been published 71.198.184.231 (talk) 22:23, 23 April 2009 (UTC)
replies: text: thanks, i put the first one and the original reference (for anyone who can find it!) in the article. -3mta3 (talk) 08:36, 24 April 2009 (UTC)
thread_title:  Title 

message: I am not sure how to go about adding this, but I think this article should reference Kanungo et al's kd-tree filtering algorithm (http://www.cs.umd.edu/~mount/Papers/pami02.pdf). It speeds a naive implementation up by a couple orders of magnitude. 71.198.184.231 (talk) 22:32, 23 April 2009 (UTC)
replies: text: There is a reference to it (see K-means_clustering#Notes) however no description of what it does. Feel free to expand. -3mta3 (talk) 08:15, 24 April 2009 (UTC)
replies: text: Ok, I expanded the variations section some -- I'm not super-sure my summary of the last one is good since only the abstract is online. I hadn't heard of it before. 71.198.184.231 (talk) 16:43, 24 April 2009 (UTC)
thread_title:  Filtering algorithm 

message: I think it is somewhat misleading to say using variance and Euclidean distance are drawbacks of k-means, and it's very misleading to say this is what makes k-means fast. These choices are what makes k-means *well defined*. What makes it *fast* is that it does not attempt to find the best clustering even according to its own definitions. Any clustering algorithm has to implicitly define what it means by a "good" clustering, and k-means's choice of Euclidean distance + variance is not obviously better or worse than other choices. So yes, if people use k-means assuming it has a human-like understanding of when two unknown objects are similar, they will be disappointed. But, no algorithm does this, and there is certainly no standard belief that an algorithm ever will be able to do this. Meanwhile, the description implies there is a clear better metric which k-means doesn't use for speed reasons - I do not think this is true and without real evidence, I do not see how the claim belongs on wikipedia in its current form.198.161.30.159 (talk) 06:29, 30 June 2009 (UTC)
replies: text: I would add that as I understand it k-means clustering as a method is independent of any particular distance metric, e.g.  manhattan distance would be a reasonable choice in some circumstances. --The locster (talk) 16:55, 2 September 2009 (UTC)
thread_title:  A nitpick 

message: As far as I have understood, the last frame is not the result of the algorithm. I think that the result should be a part of the demonstration, too.
Bubla (talk) 11:25, 24 August 2009 (UTC)
replies: text: Also, the caption to the first image says "initial means are randomly selected from the data set" but the means from the first image are not present as data points in the subsequent images - implying they're not part of the data set. 78.86.200.205 (talk) 10:15, 31 January 2012 (UTC)
replies: text:  I think it would be better to assume the problem is in the subsequent images, not the description (see section titled "Error in demonstration images"). — Preceding unsigned comment added by 71.56.157.70 (talk) 16:54, 27 August 2012 (UTC)
thread_title:  The demonstration is complete? 

message: As I understand it k-means clustering can be used with a variety of distanec metrics, it is common to use Euclidean distance but other metrics can be used, e.g. Manhattan. It would be nice to see an overview of some metrics that are typically used and what their relative pros and cons are.
Following on from distance metrics, some explanation of how to calculate centroids for non-euclidean distance metrics would be required, e.g. manhattan distance centroids are calculated using the median of each coordinate element rather than the mean. Such explanations probably belong elsewhere with a pointer from here but in the absence of explanations elsewhere this page is better than nowhere for now.--The locster (talk) 17:04, 2 September 2009 (UTC)
thread_title:  Choice of distance metric  

message: It's my understanding that it is possible for k-means clustering to result in empty clusters after an iteration of the algorithm. Some discussion of this issue is warranted. E.g. this is potentially more of a problem in higher dimensional spaces and for certain distance metrics. Are there algorithm variants that avoid empty clusters altogether? And what are the strategies for dealign with empty clusters when they are encountered, e.g. split the largest cluster into two, or the cluster with the highest variance, etc.--The locster (talk) 17:09, 2 September 2009 (UTC)
thread_title:  Empty clusters 

message: Is this algorithm guaranteed to converge? 140.180.6.33 (talk)  —Preceding undated comment added 07:20, 9 December 2009 (UTC)
thread_title:  Convergence guarantees? 

message: Since Lloyd's algorithm is the standard algorithm for k-means these days, and essential to explaining k-means, we might just as well merge the two articles.
In fact, I'd even suggest to make Lloyd's algorithm simply a redirect to the appropriate section of k-means where it is explained.
However, there seems to be some other variant (not so much for clustering?) of Lloyd discussed in that other article, too. --Chire (talk) 14:28, 26 August 2013 (UTC)
replies: text: Oppose. Lloyd's algorithm is also used for constructing Centroidal Voronoi tessellations and for smoothing finite element meshes. Since it has such varied applications, not all of which have anything to do with clustering, I feel that it deserves to be a separate article. And in addition, there are other algorithms than Lloyd for K-means, so the two subjects are not the same even in that context. —David Eppstein (talk) 16:09, 26 August 2013 (UTC)

text: Oppose. This was proposed in 2007, see the Lloyd's algorithm talk page for the old discussion. I feel the important distinction is that k-means (at least the k-means article) is an exclusively discrete treatment while Lloyd's algorithm has importance in a continuous setting. The Lloyd's algorithm article probably needs improvement to better reflect this: the third paragraph of that page discusses the discrete variant while the subsequent description is more of the continuous language. RuppertsAlgorithm (talk) 14:03, 27 August 2013 (UTC)

text: Oppose and disagree with the statement that Lloyd's algorithm is the standard algorithm. The default method in R is Hartigan and Wong. It is completely misleading to confuse a general method with one of several algorithms by which one can implement the method. Mathstat (talk) 15:44, 27 August 2013 (UTC)
replies: text:  For those who do not have R installed: from the R manual for kmeans: "The algorithm of Hartigan and Wong (1979) is used by default. Note that some authors use k-means to refer to a specific algorithm rather than the general method: most commonly the algorithm given by MacQueen (1967) but sometimes that given by Lloyd (1957) and Forgy (1965). The Hartigan–Wong algorithm generally does a better job than either of those, but trying several random starts (nstart> 1) is often recommended." Mathstat (talk) 12:34, 28 August 2013 (UTC)
thread_title:  Merge proposal 

message: I was surprised to find no mention of K-means++ in the article and came to the talk page to suggest it only to find that there's been some previous edit war concerning too much emphasis on K-means++. I believe the eradication of K-means++ from this article has gone a bit too far. K-means++ *is* K-means, just with a more sensible starting point. There ought to be a happy medium that can be found.  — Preceding unsigned comment added by 24.18.238.89 (talk) 16:24, 23 October 2013 (UTC)
replies: text: K-means++ has its own article. There is no need to discuss it again here. It's properly listed in the K-means_clustering#Variations long list of variations. There has also been critizism that k-means++ was overhyped, and in many situations does not improve the results / runtime that much; while the seeding process itself itself is also quite expensive: it requires O(n*k) distance computations. --Chire (talk) 15:27, 25 October 2013 (UTC)
thread_title:  K-means++ 

message: The article on the one hand says that the problem is NP-hard, and on the other hand, it states the following:
"If k and d (the dimension) are fixed, the problem can be exactly solved in time O(n^{dk+1}\log{n}), where n is the number of entities to be clustered".
I assume that NP-hardness is w.r.t. to the number of entities to be clustered (i.e., n), but the latter statement implies polynomial time.
It would be great if an expert could clarify which statement is true. (I suspect the statement quoted above is false and that the paper from which it is taken actually claims something else...)
Cervisiarius (talk) 17:47, 17 March 2016 (UTC)
replies: text: I don't know if that statement is true. Complexity analysis of such problems is hard. What if you chose e.g. k = \sqrt{n}, which is a common practise to reduce data set size? P.S. new discussions should be added to the end. HelpUsStopSpam (talk) 20:47, 17 March 2016 (UTC)
thread_title:  Confusion about computational complexity 

message: "The Forgy method tends to spread the initial means out, while Random Partition places all of them close to the center of the data set."
Seems wrong to me, logically when first adding points randomly to clusters and then calculating the means of those clusters, I'd expect those means to lie close to the center, whereas when picking 'means' randomly from the set they would be spread out.
But as I'm no expert on the algorithm or terminology I don't really dare change it by myself.
145.15.244.14 (talk) 19:33, 2 November 2017 (UTC)
replies: text: random partition = every point is randomly assigned partition 1..k => expected value of each partition is the overall data center. Seems to be correct?  — Preceding unsigned comment added by 77.177.197.101 (talk) 21:43, 2 November 2017 (UTC)
thread_title:  Shouldn't these be the other way round? 

message: Out of the blue, the second paragraph of the intro talks about "both algorithms", even though no algorithms have been mentioned before. The third paragraph then continues with "the algorithm", and again it is unclear what algorithm is being talked about. AxelBoldt (talk) 14:20, 22 December 2017 (UTC)
thread_title:  Can't understand intro 

message: This article makes no mention of usage of k-means for color extraction of images (probably because it is a non-notable concept?) Draheinsunvale (talk) 00:25, 30 December 2021 (UTC)
thread_title:  Using k-meams for dominant colors? 

message: added a not-so-pseudo pseudocode (actually it is closer to real code than to pseudocode), thank you. However, I would suggest the following change: instead of using (more expensive, because of reallocations) lists for each cluster, use a labels array (static allocation). No change in cluster labels is a convergence criterion that does not suffer from numerical precision issues, and does not have the hidden complexity of comparing the lists of centers numerically in the current code, and the label array makes the partitioning property of k-means clear. Maybe also make the code more pseudo code, to avoid that people think they can just copy this into their programs... Pseudocode should be more abstract IMHO, focus on intent instead of syntax. 2A01:C22:BC1E:A700:D040:CD72:CAA9:1C20 (talk) 11:14, 20 April 2024 (UTC)
thread_title:  Pseudocode