FIX numerical stability issues on 32 bit platform

The gini index and the mean square error (variance) might differ due
to numerical instability. Since those instabilities mainly occurs at
high tree depth, we restrict the maximal depth.