TST: linalg: better rtol choice