fixed numerical rounding error