Repository: scipy/scipy
Pull request: Bug: setting iprint=0 hides all output from fmin_l_bfgs_b, but it should print convergence info from final iteration.
This pull request fixes #13321.

In `scipy.optimize.fmin_l_bfgs_b`, the `disp` argument was handled by setting: 
```python
 if disp is None:
    disp = iprint
```
 However, both `disp` and `iprint` are fed into `scipy.optimize._minimize_lbfgsb`, which handles the `disp` argument again, with this code:
```python
if disp is not None:	 
    if disp == 0:	 
         iprint = -1
    else:
         iprint = disp
```
If `iprint=0`, then `disp` is set to 0 in `scipy.optimize.fmin_l_bfgs_b`, which again sets `iprint=-1` in `scipy.optimize._minimize_lbfgsb`, supressing all convergence reporting, despite us asking for it. By removing the relevant code in `scipy.optimize.fmin_l_bfgs_b`, we fix this behaviour, and we believe that should be ok, since `disp` is not used anywhere else in `scipy.optimize.fmin_l_bfgs_b`, except for feeding it to `scipy.optimize._minimize_lbfgsb`.

The code below demonstrates that the new behaviour is in line with the documentation.

```python
from itertools import product
import numpy as np
import scipy.optimize as optimize
from scipy.optimize.optimize import rosen, rosen_der


iprints = [0, 1]
disps = [None, 0, 1]
for iprint, disp in product(iprints, disps):
    print(f"iprint={iprint}, disp={disp}")
    print("="*80)
    result = optimize.fmin_l_bfgs_b(rosen, np.array([4, 3]), rosen_der, iprint=iprint, disp=disp, bounds=((3, 5), (3, 5)))
    print("-"*80)
```
Output:
```
iprint=0, disp=None
================================================================================
RUNNING THE L-BFGS-B CODE

           * * *

Machine precision = 2.220D-16
 N =            2     M =           10

           * * *

Tit   = total number of iterations
Tnf   = total number of function evaluations
Tnint = total number of segments explored during Cauchy searches
Skip  = number of BFGS updates skipped
Nact  = number of active bounds at final generalized Cauchy point
Projg = norm of the final projected gradient
F     = final function value

           * * *

   N    Tit     Tnf  Tnint  Skip  Nact     Projg        F
    2      1      2      2     0     2   0.000D+00   1.604D+03

CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL            
--------------------------------------------------------------------------------
iprint=0, disp=0
================================================================================
--------------------------------------------------------------------------------
iprint=0, disp=1
================================================================================
RUNNING THE L-BFGS-B CODE

           * * *

Machine precision = 2.220D-16
 N =            2     M =           10

At X0         1 variables are exactly at the bounds

At iterate    0    f=  1.69090D+04    |proj g|=  2.00000D+00

At iterate    1    f=  1.60400D+03    |proj g|=  0.00000D+00

           * * *

Tit   = total number of iterations
Tnf   = total number of function evaluations
Tnint = total number of segments explored during Cauchy searches
Skip  = number of BFGS updates skipped
Nact  = number of active bounds at final generalized Cauchy point
Projg = norm of the final projected gradient
F     = final function value

           * * *

   N    Tit     Tnf  Tnint  Skip  Nact     Projg        F
    2      1      2      2     0     2   0.000D+00   1.604D+03
  F =   1604.0000000000000     

CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL            
--------------------------------------------------------------------------------
iprint=1, disp=None
================================================================================
RUNNING THE L-BFGS-B CODE

           * * *

Machine precision = 2.220D-16
 N =            2     M =           10

At X0         1 variables are exactly at the bounds

At iterate    0    f=  1.69090D+04    |proj g|=  2.00000D+00

At iterate    1    f=  1.60400D+03    |proj g|=  0.00000D+00

           * * *

Tit   = total number of iterations
Tnf   = total number of function evaluations
Tnint = total number of segments explored during Cauchy searches
Skip  = number of BFGS updates skipped
Nact  = number of active bounds at final generalized Cauchy point
Projg = norm of the final projected gradient
F     = final function value

           * * *

   N    Tit     Tnf  Tnint  Skip  Nact     Projg        F
    2      1      2      2     0     2   0.000D+00   1.604D+03
  F =   1604.0000000000000     

CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL            
--------------------------------------------------------------------------------
iprint=1, disp=0
================================================================================
--------------------------------------------------------------------------------
iprint=1, disp=1
================================================================================
RUNNING THE L-BFGS-B CODE

           * * *

Machine precision = 2.220D-16
 N =            2     M =           10

At X0         1 variables are exactly at the bounds

At iterate    0    f=  1.69090D+04    |proj g|=  2.00000D+00

At iterate    1    f=  1.60400D+03    |proj g|=  0.00000D+00

           * * *

Tit   = total number of iterations
Tnf   = total number of function evaluations
Tnint = total number of segments explored during Cauchy searches
Skip  = number of BFGS updates skipped
Nact  = number of active bounds at final generalized Cauchy point
Projg = norm of the final projected gradient
F     = final function value

           * * *

   N    Tit     Tnf  Tnint  Skip  Nact     Projg        F
    2      1      2      2     0     2   0.000D+00   1.604D+03
  F =   1604.0000000000000     

CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL            
--------------------------------------------------------------------------------
```
We see that the `disp` option overrides the `iprint` option when set, but not otherwise.

Problem statement:
Bug: setting iprint=0 hides all output from fmin_l_bfgs_b, but it should print convergence info from final iteration.
`scipy.optimize.fmin_l_bfgs_b` provides an option to print a report after the optimisation procedure with the keyword argument `iprint=0`. However, this option does not work, as demonstrated by the code below. This bug is likely caused by how the display option (required for the minimize wrapper) is implemented, as it modifies the same behaviour as `iprint`, but in a different manner. If `display=0`, then all output from `scipy.optimize.fmin_l_bfgs_b` is suppressed, whereas we should set `iprint=-1` for the same behaviour.

The `display` argument is handled in both `scipy.optimize.fmin_l_bfgs_b` and `scipy.optimize._minimize_lbfgsb`, and this seems to cause the problem. It should be sufficient to handle the `display` argument in `scipy.optimize._minimize_lbfgsb`.

We can provide a pull request to fix this bug.

#### Reproducing code example:
This code should display convergence information of the final iteration, but it does not.
```python
import scipy.optimize as optimize
from scipy.optimize.optimize import rosen, rosen_der
import numpy as np

result = optimize.fmin_l_bfgs_b(rosen, np.array([4, 3]), rosen_der, iprint=0, bounds=((3, 5), (3, 5)))
```

#### Scipy/Numpy/Python version information:
```
import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)
```
--> 1.5.2 1.18.5 sys.version_info(major=3, minor=8, micro=6, releaselevel='final', serial=0)

Hints from issue discussion:

Commit message: TST: optimize.fmin_l_bfgs_b: test console output behavior

Patch to review:
diff --git a/scipy/optimize/_lbfgsb_py.py b/scipy/optimize/_lbfgsb_py.py
index a39e179f6aa3..c030f80dd7d3 100644
--- a/scipy/optimize/_lbfgsb_py.py
+++ b/scipy/optimize/_lbfgsb_py.py
@@ -183,8 +183,6 @@ def fmin_l_bfgs_b(func, x0, fprime=None, args=(),
         jac = fprime
 
     # build options
-    if disp is None:
-        disp = iprint
     opts = {'disp': disp,
             'iprint': iprint,
             'maxcor': m,
diff --git a/scipy/optimize/tests/test_optimize.py b/scipy/optimize/tests/test_optimize.py
index 31881582b33f..9190cd20fb38 100644
--- a/scipy/optimize/tests/test_optimize.py
+++ b/scipy/optimize/tests/test_optimize.py
@@ -1065,6 +1065,39 @@ def custmin(fun, x0, args=(), maxfev=None, stepsize=0.1,
                                 options=dict(stepsize=0.05))
         assert_allclose(res.x, 1.0, rtol=1e-4, atol=1e-4)
 
+    def test_gh13321(self, capfd):
+        # gh-13321 reported issues with console output in fmin_l_bfgs_b;
+        # check that iprint=0 works.
+        kwargs = {'func': optimize.rosen, 'x0': [4, 3],
+                  'fprime': optimize.rosen_der, 'bounds': ((3, 5), (3, 5))}
+
+        # "L-BFGS-B" is always in output; should show when iprint >= 0
+        # "At iterate" is iterate info; should show when iprint >= 1
+
+        optimize.fmin_l_bfgs_b(**kwargs, iprint=-1)
+        out, _ = capfd.readouterr()
+        assert "L-BFGS-B" not in out and "At iterate" not in out
+
+        optimize.fmin_l_bfgs_b(**kwargs, iprint=0)
+        out, _ = capfd.readouterr()
+        assert "L-BFGS-B" in out and "At iterate" not in out
+
+        optimize.fmin_l_bfgs_b(**kwargs, iprint=1)
+        out, _ = capfd.readouterr()
+        assert "L-BFGS-B" in out and "At iterate" in out
+
+        # `disp is not None` overrides `iprint` behavior
+        # `disp=0` should suppress all output
+        # `disp=1` should be the same as `iprint = 1`
+
+        optimize.fmin_l_bfgs_b(**kwargs, iprint=1, disp=False)
+        out, _ = capfd.readouterr()
+        assert "L-BFGS-B" not in out and "At iterate" not in out
+
+        optimize.fmin_l_bfgs_b(**kwargs, iprint=-1, disp=True)
+        out, _ = capfd.readouterr()
+        assert "L-BFGS-B" in out and "At iterate" in out
+
     def test_gh10771(self):
         # check that minimize passes bounds and constraints to a custom
         # minimizer without altering them.


Code review comments:
diff_hunk: @@ -1065,6 +1065,39 @@ def custmin(fun, x0, args=(), maxfev=None, stepsize=0.1,
                                 options=dict(stepsize=0.05))
         assert_allclose(res.x, 1.0, rtol=1e-4, atol=1e-4)
 
+    def test_gh13321(self, capfd):
path: scipy/optimize/tests/test_optimize.py
text: @user2:
```suggestion
    @user1.mark.xfail(reason="output not reliable on all platforms")
    def test_gh13321(self, capfd):
```