fix numerical errors