calculation precision test