solve for potential rounding error