Repository: scipy/scipy

Issue:
different default atol, btol for lsqr, lsmr
The default `atol` and `btol` for [lsqr](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.lsqr.html) is `1.0e-8`, but only `1.0e-6` for [lsmr](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.lsmr.html).


Discussion:
Hi, thanks. But should it be the same values? Is there underlying assumptions supporting that? What about specifics of the current implementation?
> But should it be the same values?

From the docstring of lsmr:
> If both tolerances are 1.0e-6 (say), the final norm(r) should be accurate to about 6 digits.

and lsqr:
> If both are 1.0e-9 (say), the final residual norm should be accurate to about 9 digits.

When tossing the same problem at lsqr and lsmr, I think the expectation should be that they (try to) solve the problem up to the same default accuracy. Note they they both measure the same thing, namely the residual norm.
Ok, I actually checked the paper of LSMR (https://arxiv.org/pdf/1006.0758.pdf) and in their comparison they say that they used the same tolerances. 

> 3.6. Practical stopping criteria. For LSMR we use the same stopping rules
as LSQR [16], involving dimensionless quantities ATOL, BTOL, CONLIM

Still, in the end the tolerance depends on the problem at hand and one should adjust it. The issue with changing default values is that we could potentially break lot of codes. Maybe we could increase the tolerance of LSMR to 1e9 to match the one of LSQR. I will let another reviewer with more background here judge.
> Still, in the end the tolerance depends on the problem at hand and one should adjust it.

Then it shouldn't have a default value, right? I actually think this is a good idea, but that'd certainly break people's code.

>  depends on the problem

Indeed, so you plug the same problem into these two different methods, by default you should expect it to be solved with the same tolerance.

> Maybe we could increase the tolerance of LSMR to 1e9 to match the one of LSQR. I will let another reviewer with more background here judge.

:+1: 

Resolving patch:
diff --git a/scipy/sparse/linalg/isolve/lsmr.py b/scipy/sparse/linalg/isolve/lsmr.py
--- a/scipy/sparse/linalg/isolve/lsmr.py
+++ b/scipy/sparse/linalg/isolve/lsmr.py
@@ -23,7 +23,7 @@
 from math import sqrt
 from scipy.sparse.linalg.interface import aslinearoperator
 
-from .lsqr import _sym_ortho
+from scipy.sparse.linalg.isolve.lsqr import _sym_ortho
 
 
 def lsmr(A, b, damp=0.0, atol=1e-6, btol=1e-6, conlim=1e8,
@@ -53,16 +53,16 @@ def lsmr(A, b, damp=0.0, atol=1e-6, btol=1e-6, conlim=1e8,
              ||(0)   (damp*I) ||_2
 
         where damp is a scalar.  If damp is None or 0, the system
-        is solved without regularization.
+        is solved without regularization. Default is 0.
     atol, btol : float, optional
         Stopping tolerances. `lsmr` continues iterations until a
         certain backward error estimate is smaller than some quantity
         depending on atol and btol.  Let ``r = b - Ax`` be the
         residual vector for the current approximate solution ``x``.
-        If ``Ax = b`` seems to be consistent, ``lsmr`` terminates
+        If ``Ax = b`` seems to be consistent, `lsmr` terminates
         when ``norm(r) <= atol * norm(A) * norm(x) + btol * norm(b)``.
-        Otherwise, lsmr terminates when ``norm(A^H r) <=
-        atol * norm(A) * norm(r)``.  If both tolerances are 1.0e-6 (say),
+        Otherwise, `lsmr` terminates when ``norm(A^H r) <=
+        atol * norm(A) * norm(r)``.  If both tolerances are 1.0e-6 (default),
         the final ``norm(r)`` should be accurate to about 6
         digits. (The final ``x`` will usually have fewer correct digits,
         depending on ``cond(A)`` and the size of LAMBDA.)  If `atol`
@@ -79,16 +79,16 @@ def lsmr(A, b, damp=0.0, atol=1e-6, btol=1e-6, conlim=1e8,
         `conlim` should be less than 1.0e+8. If `conlim` is None, the
         default value is 1e+8.  Maximum precision can be obtained by
         setting ``atol = btol = conlim = 0``, but the number of
-        iterations may then be excessive.
+        iterations may then be excessive. Default is 1e8.
     maxiter : int, optional
         `lsmr` terminates if the number of iterations reaches
         `maxiter`.  The default is ``maxiter = min(m, n)``.  For
         ill-conditioned systems, a larger value of `maxiter` may be
-        needed.
+        needed. Default is False.
     show : bool, optional
-        Print iterations logs if ``show=True``.
+        Print iterations logs if ``show=True``. Default is False.
     x0 : array_like, shape (n,), optional
-        Initial guess of ``x``, if None zeros are used.
+        Initial guess of ``x``, if None zeros are used. Default is None.
 
         .. versionadded:: 1.0.0
         
@@ -152,7 +152,7 @@ def lsmr(A, b, damp=0.0, atol=1e-6, btol=1e-6, conlim=1e8,
     >>> istop
     0
     >>> x
-    array([ 0.,  0.])
+    array([0., 0.])
 
     The stopping code `istop=0` returned indicates that a vector of zeros was
     found as a solution. The returned solution `x` indeed contains `[0., 0.]`.
@@ -199,13 +199,13 @@ def lsmr(A, b, damp=0.0, atol=1e-6, btol=1e-6, conlim=1e8,
         b = b.squeeze()
 
     msg = ('The exact solution is x = 0, or x = x0, if x0 was given  ',
-         'Ax - b is small enough, given atol, btol                  ',
-         'The least-squares solution is good enough, given atol     ',
-         'The estimate of cond(Abar) has exceeded conlim            ',
-         'Ax - b is small enough for this machine                   ',
-         'The least-squares solution is good enough for this machine',
-         'Cond(Abar) seems to be too large for this machine         ',
-         'The iteration limit has been reached                      ')
+           'Ax - b is small enough, given atol, btol                  ',
+           'The least-squares solution is good enough, given atol     ',
+           'The estimate of cond(Abar) has exceeded conlim            ',
+           'Ax - b is small enough for this machine                   ',
+           'The least-squares solution is good enough for this machine',
+           'Cond(Abar) seems to be too large for this machine         ',
+           'The iteration limit has been reached                      ')
 
     hdg1 = '   itn      x(1)       norm r    norm Ar'
     hdg2 = ' compatible   LS      norm A   cond A'
diff --git a/scipy/sparse/linalg/isolve/lsqr.py b/scipy/sparse/linalg/isolve/lsqr.py
--- a/scipy/sparse/linalg/isolve/lsqr.py
+++ b/scipy/sparse/linalg/isolve/lsqr.py
@@ -93,7 +93,7 @@ def _sym_ortho(a, b):
     return c, s, r
 
 
-def lsqr(A, b, damp=0.0, atol=1e-8, btol=1e-8, conlim=1e8,
+def lsqr(A, b, damp=0.0, atol=1e-6, btol=1e-6, conlim=1e8,
          iter_lim=None, show=False, calc_var=False, x0=None):
     """Find the least-squares solution to a large, sparse, linear system
     of equations.
@@ -125,12 +125,25 @@ def lsqr(A, b, damp=0.0, atol=1e-8, btol=1e-8, conlim=1e8,
     b : array_like, shape (m,)
         Right-hand side vector ``b``.
     damp : float
-        Damping coefficient.
+        Damping coefficient. Default is 0.
     atol, btol : float, optional
-        Stopping tolerances. If both are 1.0e-9 (say), the final
-        residual norm should be accurate to about 9 digits.  (The
-        final x will usually have fewer correct digits, depending on
-        cond(A) and the size of damp.)
+        Stopping tolerances. `lsqr` continues iterations until a
+        certain backward error estimate is smaller than some quantity
+        depending on atol and btol.  Let ``r = b - Ax`` be the
+        residual vector for the current approximate solution ``x``.
+        If ``Ax = b`` seems to be consistent, `lsqr` terminates
+        when ``norm(r) <= atol * norm(A) * norm(x) + btol * norm(b)``.
+        Otherwise, `lsqr` terminates when ``norm(A^H r) <=
+        atol * norm(A) * norm(r)``.  If both tolerances are 1.0e-6 (default),
+        the final ``norm(r)`` should be accurate to about 6
+        digits. (The final ``x`` will usually have fewer correct digits,
+        depending on ``cond(A)`` and the size of LAMBDA.)  If `atol`
+        or `btol` is None, a default value of 1.0e-6 will be used.
+        Ideally, they should be estimates of the relative error in the
+        entries of ``A`` and ``b`` respectively.  For example, if the entries
+        of ``A`` have 7 correct digits, set ``atol = 1e-7``. This prevents
+        the algorithm from doing unnecessary work beyond the
+        uncertainty of the input data.
     conlim : float, optional
         Another stopping tolerance.  lsqr terminates if an estimate of
         ``cond(A)`` exceeds `conlim`.  For compatible systems ``Ax =
@@ -138,15 +151,15 @@ def lsqr(A, b, damp=0.0, atol=1e-8, btol=1e-8, conlim=1e8,
         least-squares problems, conlim should be less than 1.0e+8.
         Maximum precision can be obtained by setting ``atol = btol =
         conlim = zero``, but the number of iterations may then be
-        excessive.
+        excessive. Default is 1e8.
     iter_lim : int, optional
         Explicit limitation on number of iterations (for safety).
     show : bool, optional
-        Display an iteration log.
+        Display an iteration log. Default is False.
     calc_var : bool, optional
         Whether to estimate diagonals of ``(A'A + damp^2*I)^{-1}``.
     x0 : array_like, shape (n,), optional
-        Initial guess of x, if None zeros are used.
+        Initial guess of x, if None zeros are used. Default is None.
 
         .. versionadded:: 1.0.0
 


Test patch: