Repository: scipy/scipy

Issue:
better documentation for 'return_all' option in minimize(Nelder Mead)
<!-- 
Thank you for taking the time to file a bug report. 
Please fill in the fields below, deleting the sections that 
don't apply to your issue. You can view the final output
by clicking the preview button above.
Note: This is a comment, and won't appear in the output.
-->

My issue is about ... dummy 'return_all' option in scipy.optimize.minimize(method=’Nelder-Mead’)
Basically, the **minimize** function returns an object of class *OptimizeResult*. This has no attribute which would enable it to return the function values at each iteration. This in contrast with the corresponding `retall` parameter in  scipy.optimize.fmin which **minimize** is proposed to replace.
#### Reproducing code example:
<!-- 
If you place your code between the triple backticks below, 
it will be rendered as a code block. 
-->

```
from scipy.optimize import minimize, rosen, rosen_der
from numpy import array
x0 = array([1.3, 0.7, 0.8, 1.9, 1.2])
res = minimize(rosen, x0, method='Nelder-Mead', 
               options={'return_all': False},
               tol=1e-6)
dir(res)
```
#### output message:
```
['final_simplex', 'fun', 'message', 'nfev', 'nit', 'status', 'success', 'x']

```
while 
```
res = minimize(rosen, x0, method='Nelder-Mead', 
               options={'return_all': True},
               tol=1e-6)
dir(res)
```
#### output message:
```
['allvecs',
 'final_simplex',
 'fun',
 'message',
 'nfev',
 'nit',
 'status',
 'success',
 'x']
```
There seems to be no documentation on allvecs, other special attributes anywhere.
At least not on [OptimizeResult docs](https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.optimize.OptimizeResult.html)
<!-- If any, paste the *full* error message inside a code block
as above (starting from line Traceback)
-->



#### Scipy/Numpy/Python version information:
Scipy 1.4.1
Numpy 1.18.1
Python  sys.version_info(major=3, minor=6, micro=10, releaselevel='final', serial=0)
<!-- You can simply run the following and paste the result in a code block
```
import sys, scipy, numpy; print(scipy.__version__, numpy.__version__, sys.version_info)
```
-->


Discussion:

Resolving patch:
diff --git a/scipy/optimize/optimize.py b/scipy/optimize/optimize.py
--- a/scipy/optimize/optimize.py
+++ b/scipy/optimize/optimize.py
@@ -569,6 +569,9 @@ def _minimize_neldermead(func, x0, args=(), callback=None,
         variables, if neither `maxiter` or `maxfev` is set. If both
         `maxiter` and `maxfev` are set, minimization will stop at the
         first reached.
+    return_all : bool, optional
+        Set to True to return a list of the best solution at each of the
+        iterations.
     initial_simplex : array_like of shape (N + 1, N)
         Initial simplex. If given, overrides `x0`.
         ``initial_simplex[j,:]`` should contain the coordinates of
@@ -1071,6 +1074,9 @@ def _minimize_bfgs(fun, x0, args=(), jac=None, callback=None,
     eps : float or ndarray
         If `jac is None` the absolute step size used for numerical
         approximation of the jacobian via forward differences.
+    return_all : bool, optional
+        Set to True to return a list of the best solution at each of the
+        iterations.
     finite_diff_rel_step : None or array_like, optional
         If `jac in ['2-point', '3-point', 'cs']` the relative step size to
         use for numerical approximation of the jacobian. The absolute step
@@ -1392,6 +1398,9 @@ def _minimize_cg(fun, x0, args=(), jac=None, callback=None,
     eps : float or ndarray
         If `jac is None` the absolute step size used for numerical
         approximation of the jacobian via forward differences.
+    return_all : bool, optional
+        Set to True to return a list of the best solution at each of the
+        iterations.
     finite_diff_rel_step : None or array_like, optional
         If `jac in ['2-point', '3-point', 'cs']` the relative step size to
         use for numerical approximation of the jacobian. The absolute step
@@ -1654,6 +1663,9 @@ def _minimize_newtoncg(fun, x0, args=(), jac=None, hess=None, hessp=None,
         Maximum number of iterations to perform.
     eps : float or ndarray
         If `hessp` is approximated, use this value for the step size.
+    return_all : bool, optional
+        Set to True to return a list of the best solution at each of the
+        iterations.
     """
     _check_unknown_options(unknown_options)
     if jac is None:
@@ -2711,6 +2723,9 @@ def _minimize_powell(func, x0, args=(), callback=None,
         first reached.
     direc : ndarray
         Initial set of direction vectors for the Powell method.
+    return_all : bool, optional
+        Set to True to return a list of the best solution at each of the
+        iterations.
 
     """
     _check_unknown_options(unknown_options)


Test patch: