test for rounding errors in stochastic algorithm