We probably want the accuracy to be somewhat closer to machine precision.
For scipy.special, I've held 1e-10 relative error as sort of a precision target.