Repository: scipy/scipy

Issue:
scipy optimize:  most methods do not return 'nit' in the result object
Please remove nit from the result object documentation.   In general, there seem to be many API inconsistencies and documentation errors.  Would be nice to fix.  Found confirmation in source.

The inconsistency is that some optimize methods return a dict and others return a result object.  No particular reason for the inconsistency.   A good cleanup to do.



Discussion:
All optimizers in minimize() appear to return OptimizeResult. Please
specify which one does not.
Please specify API inconsistencies and documentation errors.

optimize result is documented to have nit, which is number of iterations.  but, nit is not a member of optimize result.
- Lewis

On Aug 3, 2014, at 3:11 PM, "Pauli Virtanen" <notifications@github.com<mailto:notifications@github.com>> wrote:

All optimizers in minimize() appear to return OptimizeResult. Please
specify which one does not.
Please specify API inconsistencies and documentation errors.

—
Reply to this email directly or view it on GitHubhttps://github.com/scipy/scipy/issues/3855#issuecomment-51004918.

It is true that some optimizers (e.g. bfgs, cg, newton-cg, cobyla and maybe others) do not set the `nit` attribute of the optimize result. Maybe the docstring of `OptimizeResult` could be improved, saying that attributes listed may not always be present depending on the underlying solver.

It’s fine.  I thought the idea of OptimizeResult was that it was the same api across the methods.  Apparently not.  The documentation of each method makes it seem that OptimizeResult is the same set of outputs.  Not clear why it couldn’t be.

My reason for wanting nit is that occasionally random initialization of neural network results in a single iteration and no meaningful result.  I’d like to test nit, intercept runs with only 1 iteration, and run again.

No worries;  I’ll find something else to test—probably the resulting params are the same as the initial params in this case.

Thanks.

From: Denis Laxalde <notifications@github.com<mailto:notifications@github.com>>
Reply-To: scipy/scipy <reply@reply.github.com<mailto:reply@reply.github.com>>
Date: Tuesday, August 5, 2014 at 12:11 AM
To: scipy/scipy <scipy@noreply.github.com<mailto:scipy@noreply.github.com>>
Cc: Lewis Levin <lewis@neilson-levin.org<mailto:lewis@neilson-levin.org>>
Subject: Re: [scipy] scipy optimize: most methods do not return 'nit' in the result object (#3855)

It is true that some optimizers (e.g. bfgs, cg, newton-cg, cobyla and maybe others) do not set the nit attribute of the optimize result. Maybe the docstring of OptimizeResult could be improved, saying that attributes listed may not always be present depending on the underlying solver.

—
Reply to this email directly or view it on GitHubhttps://github.com/scipy/scipy/issues/3855#issuecomment-51158405.


Resolving patch:
diff --git a/scipy/optimize/optimize.py b/scipy/optimize/optimize.py
--- a/scipy/optimize/optimize.py
+++ b/scipy/optimize/optimize.py
@@ -928,7 +928,8 @@ def _minimize_bfgs(fun, x0, args=(), jac=None, callback=None,
 
     result = OptimizeResult(fun=fval, jac=gfk, hess_inv=Hk, nfev=func_calls[0],
                             njev=grad_calls[0], status=warnflag,
-                            success=(warnflag == 0), message=msg, x=xk)
+                            success=(warnflag == 0), message=msg, x=xk,
+                            nit=k)
     if retall:
         result['allvecs'] = allvecs
     return result
@@ -1210,7 +1211,8 @@ def _minimize_cg(fun, x0, args=(), jac=None, callback=None,
 
     result = OptimizeResult(fun=fval, jac=gfk, nfev=func_calls[0],
                             njev=grad_calls[0], status=warnflag,
-                            success=(warnflag == 0), message=msg, x=xk)
+                            success=(warnflag == 0), message=msg, x=xk,
+                            nit=k)
     if retall:
         result['allvecs'] = allvecs
     return result
@@ -1478,7 +1480,8 @@ def _minimize_newtoncg(fun, x0, args=(), jac=None, hess=None, hessp=None,
 
     result = OptimizeResult(fun=fval, jac=gfk, nfev=fcalls[0], njev=gcalls[0],
                             nhev=hcalls, status=warnflag,
-                            success=(warnflag == 0), message=msg, x=xk)
+                            success=(warnflag == 0), message=msg, x=xk,
+                            nit=k)
     if retall:
         result['allvecs'] = allvecs
     return result


Test patch:
diff --git a/scipy/optimize/tests/test_optimize.py b/scipy/optimize/tests/test_optimize.py
--- a/scipy/optimize/tests/test_optimize.py
+++ b/scipy/optimize/tests/test_optimize.py
@@ -878,5 +878,37 @@ def func2(x):
     assert_allclose(res.x, [125, 0, 0], atol=1e-10)
 
 
+class TestOptimizeResultAttributes(TestCase):
+    # Test that all minimizers return an OptimizeResult containing
+    # all the OptimizeResult attributes
+    def setUp(self):
+        self.x0 = [5, 5]
+        self.func = optimize.rosen
+        self.jac = optimize.rosen_der
+        self.hess = optimize.rosen_hess
+        self.hessp = optimize.rosen_hess_prod
+        self.bounds = [(0., 10.), (0., 10.)]
+
+    def test_attributes_present(self):
+        methods = ['Nelder-Mead', 'Powell', 'CG', 'BFGS', 'Newton-CG',
+                   'L-BFGS-B', 'TNC', 'COBYLA', 'SLSQP', 'dogleg',
+                   'trust-ncg']
+        attributes = ['nit', 'nfev', 'x', 'success', 'status', 'fun',
+                      'message']
+        skip = {'COBYLA': ['nit']}
+        for method in methods:
+            with warnings.catch_warnings():
+                warnings.simplefilter("ignore")
+                res = optimize.minimize(self.func, self.x0, method=method,
+                                        jac=self.jac, hess=self.hess,
+                                        hessp=self.hessp)
+            for attribute in attributes:
+                if method in skip and attribute in skip[method]:
+                    continue
+
+                assert_(hasattr(res, attribute))
+
+
 if __name__ == "__main__":
     run_module_suite()
+