Repository: scipy/scipy

Issue:
OptimizeResult from minimize_scalar (Brent's method) missing attributes or incorrectly documented
From the mimize_scalar docstring:

```
Returns
-------
res : OptimizeResult
    The optimization result represented as a ``OptimizeResult`` object.
    Important attributes are: ``x`` the solution array, ``success`` a
    Boolean flag indicating if the optimizer exited successfully and
    ``message`` which describes the cause of the termination. See
    `OptimizeResult` for a description of other attributes.
```

But the result does not have `success` or `message` attributes.  From the source for optimize._minimize_scalar_brent():

```
return OptimizeResult(fun=fval, x=x, nit=nit, nfev=nfev)
```

So the returned value is indeed an OptimizeResult object, but it does not have all of the "important attributes" defined.

A quick fix would be to modify the minimize_scalar docstring to correctly identify the OptimizeResults attributes that are guaranteed to be defined.  But it appears that throughout optimize, a minimal set of attributes is presumed to be defined for all optimization results, so a better fix may be to make sure that minimize_scalar always defines the "important attributes" specified in the docstring.



Discussion:
It should be possible to detect that the maximum number of iterations has been exceeded in `Brent.optimize()`. For instance, just add an `else` clause to the `while` loop setting a dedicated attribute, and retrieve it from the caller which builds the `OptimizeResult`.

With the last patch, this issue seems to have been fixed. If not, is it ok to take it and work on?
I have also noticed this issue on Basinhopping if I remember correctly, I'll see if I can replicate it later
`message` is present only in `OptimizeResult` from `bounded`:
```
import scipy.optimize
for method in ["brent", "golden", "bounded"]:
  print(method + ":")
  print(scipy.optimize.minimize_scalar(lambda x: x**2, method = method, bounds = [-10, 10]))
```
Gives:
```
brent:
     fun: 0.0
    nfev: 8
     nit: 4
 success: True
       x: 0.0
golden:
     fun: 0.0
    nfev: 818
     nit: 813
 success: True
       x: 1.5717277788484873e-162
bounded:
     fun: 0.0
 message: 'Solution found.'
    nfev: 6
  status: 0
 success: True
       x: 0.0
```

Solution should be to add `message`, since [`OptimizeResult` documentation](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.OptimizeResult.html) specifies that it will be an attribute of an `OptimizeResult`.
@MiroFurtado 
I'm not seeing a problem with `basinhopping`:
```
print(scipy.optimize.basinhopping(lambda x: x**2, x0 = [1]))
```
It has:
```
  message: 'Optimization terminated successfully.'
```
@mdhaber, the problem with the return value from `basinhopping` is that it doesn't have the `success` attribute.  This was just reported in a question on stackoverflow: https://stackoverflow.com/questions/59598917/how-to-access-success-attribute-of-optimizeresult-instance-returned-from-scipy

E.g.
```
In [2]: res = basinhopping(lambda x: x**2, x0=1)                                                             

In [3]: res.success                                                                                          
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~/mc37scipy/lib/python3.7/site-packages/scipy-1.5.0.dev0+1984f97-py3.7-macosx-10.7-x86_64.egg/scipy/optimize/optimize.py in __getattr__(self, name)
    117         try:
--> 118             return self[name]
    119         except KeyError:

KeyError: 'success'

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
<ipython-input-3-1c23ad034239> in <module>
----> 1 res.success

~/mc37scipy/lib/python3.7/site-packages/scipy-1.5.0.dev0+1984f97-py3.7-macosx-10.7-x86_64.egg/scipy/optimize/optimize.py in __getattr__(self, name)
    118             return self[name]
    119         except KeyError:
--> 120             raise AttributeError(name)
    121 
    122     __setattr__ = dict.__setitem__

AttributeError: success
```

And, as in the initial report in this issue, `minimize_scalar` still does not set the `message` attribute:

```
In [4]: res = minimize_scalar(lambda x: x**2, [-1, 3])                                                       

In [5]: res.message                                                                                          
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~/mc37scipy/lib/python3.7/site-packages/scipy-1.5.0.dev0+1984f97-py3.7-macosx-10.7-x86_64.egg/scipy/optimize/optimize.py in __getattr__(self, name)
    117         try:
--> 118             return self[name]
    119         except KeyError:

KeyError: 'message'

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
<ipython-input-5-ffaa636607ae> in <module>
----> 1 res.message

~/mc37scipy/lib/python3.7/site-packages/scipy-1.5.0.dev0+1984f97-py3.7-macosx-10.7-x86_64.egg/scipy/optimize/optimize.py in __getattr__(self, name)
    118             return self[name]
    119         except KeyError:
--> 120             raise AttributeError(name)
    121 
    122     __setattr__ = dict.__setitem__

AttributeError: message

```

This is with the development version of scipy:
```
In [6]: import scipy                                                                                         

In [7]: scipy.__version__                                                                                    
Out[7]: '1.5.0.dev0+1984f97'
```
Ah ok.
I'm not sure what an `OptimizeResult.success` attribute means in the context of `basinhopping`; it's not clear what a fail consists of.

- most of the fault conditions in setup are done by raising Exceptions rather than a success flag being set.
- if there are local minimisation failures they are noted in BasinHoppingRunner, but they don't necessarily mean that the entire run is a failure as other local minimisations may have worked.

Still, I would expect that attribute to be set.

Just setting it to `true` might be ok? Just so that the return object fulfills the `OptimizeResult`.
I think adding this line maybe useful

![image](https://user-images.githubusercontent.com/54520113/117089873-102bd480-ad57-11eb-8fdd-cbe50f119479.png)


Resolving patch:
diff --git a/scipy/optimize/_basinhopping.py b/scipy/optimize/_basinhopping.py
--- a/scipy/optimize/_basinhopping.py
+++ b/scipy/optimize/_basinhopping.py
@@ -734,6 +734,7 @@ def basinhopping(func, x0, niter=100, T=1.0, stepsize=0.5,
     res.fun = res.lowest_optimization_result.fun
     res.message = message
     res.nit = i + 1
+    res.success = res.lowest_optimization_result.success
     return res
 
 


Test patch:
diff --git a/scipy/optimize/tests/test_optimize.py b/scipy/optimize/tests/test_optimize.py
--- a/scipy/optimize/tests/test_optimize.py
+++ b/scipy/optimize/tests/test_optimize.py
@@ -2545,3 +2545,26 @@ def fquad_jac(x):
             hess = fquad_hess
         res = optimize.minimize(fquad, x0, method=meth, jac=jac, hess=hess)
         assert_allclose(res.x, np.arange(np.size(x0)), atol=2e-4)
+
+
+class TestGlobalOptimization:
+
+    def test_optimize_result_attributes(self):
+        def func(x):
+            return x ** 2
+
+        # Note that `brute` solver does not return `OptimizeResult`
+        results = [optimize.basinhopping(func, x0=1),
+                   optimize.differential_evolution(func, [(-4, 4)]),
+                   optimize.shgo(func, [(-4, 4)]),
+                   optimize.dual_annealing(func, [(-4, 4)]),
+                   ]
+
+        for result in results:
+            assert isinstance(result, optimize.OptimizeResult)
+            assert hasattr(result, "x")
+            assert hasattr(result, "success")
+            assert hasattr(result, "message")
+            assert hasattr(result, "fun")
+            assert hasattr(result, "nfev")
+            assert hasattr(result, "nit")