apache/spark — pull request #3565 by dbtsai (2014-12-03 01:22:21+00:00)
Title: [SPARK-4708][MLLib] Make k-mean runs two/three times faster with dense/sparse sample

Note that the usage of `breezeSquaredDistance` in
`org.apache.spark.mllib.util.MLUtils.fastSquaredDistance`
is in the critical path, and breezeSquaredDistance is slow.
We should replace it with our own implementation.

Here is the benchmark against mnist8m dataset.
Before
DenseVector: 70.04secs
SparseVector: 59.05secs

With this PR
DenseVector: 30.58secs
SparseVector: 21.14secs