fixed floating point rounding errors