avoiding rounding errors, added tests (refs #499)