apache/spark — pull request #568 by tmyklebu (2014-04-26 19:26:22+00:00)
Title: Micro-optimisation of ALS

This change replaces some Scala `for` and `foreach` constructs with `while` constructs. There may be a slight performance gain on the order of 1-2% when training an ALS model.

I trained an ALS model on the Movielens 10M-rating dataset repeatedly both with and without these changes. All 7 runs in both columns were done in a Scala `for` loop like this:

for (iter <- 0 to 10) {
val before = System.currentTimeMillis()
val model = ALS.train(rats, 20, 10)
val after = System.currentTimeMillis()
println("%d ms".format(after-before))
println("rmse %g".format(computeRmse(model, rats, numRatings)))
}

The timings were done on a multiuser machine, and I stopped one set of timings after 7 had been completed. It would be nice if somebody with dedicated hardware could confirm my timings.

After Before
121980 ms 122041 ms
117069 ms 117127 ms
115332 ms 117523 ms
115381 ms 117402 ms
114635 ms 116550 ms
114140 ms 114076 ms
112993 ms 117200 ms

Ratios are about 1.0005, 1.0005, 1.019, 1.0175, 1.01671, 0.99944, and 1.03723. I therefore suspect these changes make for a slight performance gain on the order of 1-2%.