scikit-learn/scikit-learn — pull request #2395 by pprett (2013-08-25 09:16:42+00:00)
Title: Gbrt stability

This PR addresses differences on 64bit and 32bit architectures and in particular #2225 .

The main problem seems to be the computation of thresholds in the tree code. Currently, we compute the split threshold as follows::

threshold = (a + b) / 2.0

While the above formular is subject is overflow this should not be an issue on most platforms (see [1] page 12 - a and b should be promoted to 80bit precision thus no overflow should occur - the div by 2 should make it fit in double prec)

The result of the above compuation differs on 32bit and 64bit. The problem can be solved by either:

a) explicitly cast a and b to double precision
b) threshold = a + (b - a ) / 2
c) threshold = a / 2 + b / 2
d) threshold = a + (b / 2 - a / 2)

the above compuations of threshold are equivalent but are subject to different rounding errors / cancelations.
I currently use solution c -- test suite passes on both 32bit and 64bit arch.

The strange thing is that AFAIK (a + b) / 2 should be numerically more accurate than other alternatives such as b) .

@glouppe @larsmans @GaelVaroquaux @amueller do you have any thoughs on that?

[1] http://hal.inria.fr/docs/00/57/66/41/PDF/computing-midpoint.pdf