Repository: scipy/scipy

Issue:
fsolve & root incorrect function-call count
The functions in scipy.optimize fsolve and root both return a wrong number function evaluations in the information they return.

Concretely, they fail to count 2 function evaluations they do for checking purposes:
from the minpack code -> es = atleast_1d(thefunc(*((x0[:numinputs],) + args)))
and another that I didn't find (probably in compiled code).

Someone already noticed a few years ago:
https://mail.scipy.org/pipermail/scipy-dev/2010-November/015765.html

So this two extra function evaluations should be counted (or at least added to the documentation as two extra function evaluations for checking purposes).

And related to this: For applications where the cost of the function evaluations is high these function evaluations should be avoided (or optional).

The code that uncovers the bug:

```
from scipy import optimize

counter=0

def zero_f(y):
    global counter
    counter += 1
    return y**2-3

counter=0
x, infodict, ier, mesg = optimize.fsolve(zero_f,100, full_output = True)
print(counter)
print ("func ev: " + str(infodict["nfev"]))

counter=0
optObject = optimize.root(zero_f,100)
print(counter)
print("func ev: " + str(optObject.nfev))
```



Discussion:
This is pretty low-hanging fruit. Here are some notes to make it easier.

This code makes it easy to see where each function call is made from.

<details>

```python3
from scipy import optimize

def zero_f(y):
    zero_f.counter += 1
    if zero_f.counter == 1:  # see stack trace after first evaluation
        raise Exception()
    return y**2-3

# zero_f.counter = 0
# x, infodict, ier, mesg = optimize.fsolve(zero_f, 100, full_output = True)
# print(zero_f.counter)
# print ("func ev: " + str(infodict["nfev"]))

zero_f.counter = 0
res = optimize.root(zero_f, 100, method='lm', options={'maxfev': 3})
print(zero_f.counter)
print("func ev: " + str(res.nfev))
```

</details>

For `root`, methods `hybr` and `lm` have the problem. (`df-sane` seems to return the correct count, and the rest don't seem to return `fev`.)

One of the calls that isn't counted is:
https://github.com/scipy/scipy/blob/656076ca6b490f587e9bd9c4cd10cb259a687c5b/scipy/optimize/_minpack_py.py#L24

The other seems to be in ` _minpack._hybrd` or `_minpack._lmdif` itself.

Option 1: Just update the number of function evaluations. 

https://github.com/scipy/scipy/blob/656076ca6b490f587e9bd9c4cd10cb259a687c5b/scipy/optimize/_minpack_py.py#L263-L272 would be a reasonable place to fix both `fsolve` and `root` `method='hybr'`. There would need to be a similar change in `leastsq` to correct `root` for `method='lm'`.

Option 2: In `root` and `fsolve`, wrap the function to count the number of evaluations. Ignore the count provided by the solvers; just populate the `OptimizeResult` with `nfev` as counted by the wrapper (e.g. like in the code above).

The tests should probably use `pytest.mark.parametrize` to loop over all the methods of `root` to make sure they're returning the correct count.

It would also be good to adjust the `maxfev` option if provided so that we don't go over by two.

I'd be happy to review a PR. I'd also be happy to submit a PR if someone else is willing to review.

Resolving patch:
diff --git a/scipy/optimize/_minpack_py.py b/scipy/optimize/_minpack_py.py
--- a/scipy/optimize/_minpack_py.py
+++ b/scipy/optimize/_minpack_py.py
@@ -155,6 +155,16 @@ def fsolve(func, x0, args=(), fprime=None, full_output=0,
     array([ True,  True])
 
     """
+    def _wrapped_func(*fargs):
+        """
+        Wrapped `func` to track the number of times
+        the function has been called.
+        """
+        _wrapped_func.nfev += 1
+        return func(*fargs)
+
+    _wrapped_func.nfev = 0
+
     options = {'col_deriv': col_deriv,
                'xtol': xtol,
                'maxfev': maxfev,
@@ -163,7 +173,9 @@ def fsolve(func, x0, args=(), fprime=None, full_output=0,
                'factor': factor,
                'diag': diag}
 
-    res = _root_hybr(func, x0, args, jac=fprime, **options)
+    res = _root_hybr(_wrapped_func, x0, args, jac=fprime, **options)
+    res.nfev = _wrapped_func.nfev
+
     if full_output:
         x = res['x']
         info = {k: res.get(k)
@@ -878,7 +890,8 @@ def curve_fit(f, xdata, ydata, p0=None, sigma=None, absolute_sigma=False,
     ...     popt, pcov = curve_fit(func, xdata, ydata, method = 'trf')
     ... except RuntimeError as e:
     ...     print(e)
-    Optimal parameters not found: The maximum number of function evaluations is exceeded.
+    Optimal parameters not found: The maximum number of function evaluations is
+    exceeded.
 
     If parameter scale is roughly known beforehand, it can be defined in
     `x_scale` argument:
diff --git a/scipy/optimize/_root.py b/scipy/optimize/_root.py
--- a/scipy/optimize/_root.py
+++ b/scipy/optimize/_root.py
@@ -199,6 +199,16 @@ def root(fun, x0, args=(), method='hybr', jac=None, tol=None, callback=None,
     >>> plt.show()
 
     """
+    def _wrapped_fun(*fargs):
+        """
+        Wrapped `func` to track the number of times
+        the function has been called.
+        """
+        _wrapped_fun.nfev += 1
+        return fun(*fargs)
+
+    _wrapped_fun.nfev = 0
+
     if not isinstance(args, tuple):
         args = (args,)
 
@@ -233,22 +243,23 @@ def root(fun, x0, args=(), method='hybr', jac=None, tol=None, callback=None,
             options.setdefault('fatol', np.inf)
 
     if meth == 'hybr':
-        sol = _root_hybr(fun, x0, args=args, jac=jac, **options)
+        sol = _root_hybr(_wrapped_fun, x0, args=args, jac=jac, **options)
     elif meth == 'lm':
-        sol = _root_leastsq(fun, x0, args=args, jac=jac, **options)
+        sol = _root_leastsq(_wrapped_fun, x0, args=args, jac=jac, **options)
     elif meth == 'df-sane':
         _warn_jac_unused(jac, method)
-        sol = _root_df_sane(fun, x0, args=args, callback=callback,
+        sol = _root_df_sane(_wrapped_fun, x0, args=args, callback=callback,
                             **options)
     elif meth in ('broyden1', 'broyden2', 'anderson', 'linearmixing',
                   'diagbroyden', 'excitingmixing', 'krylov'):
         _warn_jac_unused(jac, method)
-        sol = _root_nonlin_solve(fun, x0, args=args, jac=jac,
+        sol = _root_nonlin_solve(_wrapped_fun, x0, args=args, jac=jac,
                                  _method=meth, _callback=callback,
                                  **options)
     else:
         raise ValueError('Unknown solver %s' % method)
 
+    sol.nfev = _wrapped_fun.nfev
     return sol
 
 
@@ -291,10 +302,19 @@ def _root_leastsq(fun, x0, args=(), jac=None,
     diag : sequence
         N positive entries that serve as a scale factors for the variables.
     """
+    nfev = 0
+    def _wrapped_fun(*fargs):
+        """
+        Wrapped `func` to track the number of times
+        the function has been called.
+        """
+        nonlocal nfev
+        nfev += 1
+        return fun(*fargs)
 
     _check_unknown_options(unknown_options)
-    x, cov_x, info, msg, ier = leastsq(fun, x0, args=args, Dfun=jac,
-                                       full_output=True,
+    x, cov_x, info, msg, ier = leastsq(_wrapped_fun, x0, args=args,
+                                       Dfun=jac, full_output=True,
                                        col_deriv=col_deriv, xtol=xtol,
                                        ftol=ftol, gtol=gtol,
                                        maxfev=maxiter, epsfcn=eps,
@@ -303,6 +323,7 @@ def _root_leastsq(fun, x0, args=(), jac=None,
                          success=ier in (1, 2, 3, 4), cov_x=cov_x,
                          fun=info.pop('fvec'), method="lm")
     sol.update(info)
+    sol.nfev = nfev
     return sol
 
 


Test patch:
diff --git a/scipy/optimize/tests/test_minpack.py b/scipy/optimize/tests/test_minpack.py
--- a/scipy/optimize/tests/test_minpack.py
+++ b/scipy/optimize/tests/test_minpack.py
@@ -259,6 +259,27 @@ def test_pressure_network_no_gradient(self):
         assert_array_almost_equal(final_flows, np.ones(4))
 
 
+class TestNfev:
+    def zero_f(self, y):
+        self.nfev += 1
+        return y**2-3
+
+    @pytest.mark.parametrize('method', ['hybr', 'lm', 'broyden1',
+                                        'broyden2', 'anderson',
+                                        'linearmixing', 'diagbroyden',
+                                        'excitingmixing', 'krylov',
+                                        'df-sane'])
+    def test_root_nfev(self, method):
+        self.nfev = 0
+        solution = optimize.root(self.zero_f, 100, method=method)
+        assert solution.nfev == self.nfev
+
+    def test_fsolve_nfev(self):
+        self.nfev = 0
+        x, info, ier, mesg = optimize.fsolve(self.zero_f, 100, full_output=True)
+        assert info['nfev'] == self.nfev
+
+
 class TestLeastSq:
     def setup_method(self):
         x = np.linspace(0, 10, 40)