Repository: scipy/scipy

Issue:
Meaning of `tol` argument in `scipy.optimize.minimize` is not explained in docs
The `scipy.optimize.minimize` method offers an interface to several minimizers. It defines a `tol` argument, for which the docs say:

```
Tolerance for termination. For detailed control, use solver-specific options.
```
https://github.com/scipy/scipy/blob/master/scipy/optimize/_minimize.py#L127-L129

However, since the specific solvers don't have a `tol` argument, they don't explain it (see for example https://docs.scipy.org/doc/scipy/reference/optimize.minimize-neldermead.html#optimize-minimize-neldermead )

Instead, the only place I've been able to find how `tol` relates to the optimizer specific stopping criteria is here: https://github.com/scipy/scipy/blob/master/scipy/optimize/_minimize.py#L436-L450

Any ideas how best to resolve this?



Discussion:
I think the sentence is trying to say "use solver-specific options 
*instead* of tol".
.
The exact termination condition varies across solvers, so the most 
precise definition of tol= is "some number in range [machine-epsilon, 1] 
that controls the precision".
.
If you care what the termination condition exactly means, then 
solver-specific options need to be used. These are in general not named 
"tol".
.
The documentation can probably be clarified, but I don't think it is 
useful to give an exact definition of what "tol" means.

I don't understand, how is it more useful to have an argument with an unspecified meaning?
Because you don't care about the precise meaning. It's just some number 
that controls the relative precision.
.
If you care about the precise meaning, then you have to dig in to the 
solver-specific options.

When `tol` is specified, the minimization algorithm sets some relevant solver-specific tolerance(s) equal to `tol`. 

For the Nelder-Mead algorithm in particular, specifying `tol` sets `xatol = tol` and `fatol = tol`. Therefore the minimizer will converge when one of the following conditions are met:

1. `dx < tol and df < tol` or
2. `number_of_function_evaluations < maxfev` or 
3. `number_of_iterations < maxiter`.

As of the time of this post, what `tol` means for a given minimization algorithm is defined in scipy here: 
https://github.com/scipy/scipy/blob/c0dc7fccc53d8a8569cde5d55673fca284bca191/scipy/optimize/_minimize.py#L563

Resolving patch:
diff --git a/scipy/optimize/_minimize.py b/scipy/optimize/_minimize.py
--- a/scipy/optimize/_minimize.py
+++ b/scipy/optimize/_minimize.py
@@ -177,7 +177,9 @@ def minimize(fun, x0, args=(), method=None, jac=None, hess=None,
         be zero whereas inequality means that it is to be non-negative.
         Note that COBYLA only supports inequality constraints.
     tol : float, optional
-        Tolerance for termination. For detailed control, use solver-specific
+        Tolerance for termination. When `tol` is specified, the selected
+        minimization algorithm sets some relevant solver-specific tolerance(s)
+        equal to `tol`. For detailed control, use solver-specific
         options.
     options : dict, optional
         A dictionary of solver options. All methods accept the following


Test patch: