threshold for numerical errors in assert statements