Repository: scipy/scipy

Issue:
the meaning of 'tol' in optimize_scalar is not consistent across minimization methods
'tol' stands for relative tolerance in 'brent' and absolute tolerance in 'bounded'. Also, it is not documented whether it is relative or absolute tolerance.

Denis Laxalde suggested on the mailing list that adding support for both kinds of tolerance in the underlying solvers might not be too hard. Even if that were done, the name 'tol' does not clarify whether it is absolute or relative, and backwards-compatibility may require that 'tol' keeps its current method-dependent meaning. 

I suggest to add unambiguous parameters such as 'abs_xtol' and 'rel_xtol', which clarify not only whether the tolerance is relative or absolute, but also that the tolerance is in the function's domain and not in the function's range. Unsupported combinations of method and parameters can be rejected with a ValueError.



Discussion:
The 1d zero finders use xtol and rtol for the absolute and relative tolerance respectively.

xtol is relative error in: fmin, _minimize_neldermead, fmin_ncg, _minimize_newtoncg, _minimize_scalar_brent, _minimize_scalar_golden, fsolve, root
xtol is absolute error in: fminbound, _minimize_scalar_bounded, ridder

There does not seem to be an optimize-wide convention. 

I like the names `atol` and `rtol` as in `assert_allclose`.

In `minimize`, the naming convention is:
- `xtol`: relative tolerance for x
- `ftol`: relative tolerance for function value (compared to start?)
- `xatol`: absolute tolerance for x
- `fatol`: absolute tolerance for f

Moreover, the `tol` argument to the `minimize()` routine seems to set relative tolerance, either for x, and if that is not possible, for f.

I think the same convention should be obeyed in `minimize_scalar`.

Backward compatibility route is probably to make the changes only in the internal routines called by `minimize_scalar`, but leave the old names in place in the old (and obsolete) `fminbound` and `ridder` interfaces.


Resolving patch:
diff --git a/scipy/optimize/_minimize.py b/scipy/optimize/_minimize.py
--- a/scipy/optimize/_minimize.py
+++ b/scipy/optimize/_minimize.py
@@ -429,9 +429,6 @@ def minimize_scalar(fun, bracket=None, bounds=None, args=(),
         options.
     options : dict, optional
         A dictionary of solver options.
-            xtol : float
-                Relative error in solution `xopt` acceptable for
-                convergence.
             maxiter : int
                 Maximum number of iterations to perform.
             disp : bool
@@ -495,7 +492,12 @@ def minimize_scalar(fun, bracket=None, bounds=None, args=(),
 
     if tol is not None:
         options = dict(options)
-        options.setdefault('xtol', tol)
+        if meth == 'bounded' and 'xatol' not in options:
+            warn("Method 'bounded' does not support relative tolerance in x; "
+                 "defaulting to absolute tolerance.", RuntimeWarning)
+            options['xatol'] = tol
+        else:
+            options.setdefault('xtol', tol)
 
     if meth == 'brent':
         return _minimize_scalar_brent(fun, bracket, args, **options)
diff --git a/scipy/optimize/optimize.py b/scipy/optimize/optimize.py
--- a/scipy/optimize/optimize.py
+++ b/scipy/optimize/optimize.py
@@ -1528,7 +1528,7 @@ def fminbound(func, x1, x2, args=(), xtol=1e-5, maxfun=500,
     for auto-bracketing).
 
     """
-    options = {'xtol': xtol,
+    options = {'xatol': xtol,
                'maxiter': maxfun,
                'disp': disp}
 
@@ -1540,7 +1540,7 @@ def fminbound(func, x1, x2, args=(), xtol=1e-5, maxfun=500,
 
 
 def _minimize_scalar_bounded(func, bounds, args=(),
-                             xtol=1e-5, maxiter=500, disp=0,
+                             xatol=1e-5, maxiter=500, disp=0,
                              **unknown_options):
     _check_unknown_options(unknown_options)
     maxfun = maxiter
@@ -1572,7 +1572,7 @@ def _minimize_scalar_bounded(func, bounds, args=(),
 
     ffulc = fnfc = fx
     xm = 0.5 * (a + b)
-    tol1 = sqrt_eps * numpy.abs(xf) + xtol / 3.0
+    tol1 = sqrt_eps * numpy.abs(xf) + xatol / 3.0
     tol2 = 2.0 * tol1
 
     if disp > 2:
@@ -1644,7 +1644,7 @@ def _minimize_scalar_bounded(func, bounds, args=(),
                 fulc, ffulc = x, fu
 
         xm = 0.5 * (a + b)
-        tol1 = sqrt_eps * numpy.abs(xf) + xtol / 3.0
+        tol1 = sqrt_eps * numpy.abs(xf) + xatol / 3.0
         tol2 = 2.0 * tol1
 
         if num >= maxfun:
@@ -1653,7 +1653,7 @@ def _minimize_scalar_bounded(func, bounds, args=(),
 
     fval = fx
     if disp > 0:
-        _endprint(x, flag, fval, maxfun, xtol, disp)
+        _endprint(x, flag, fval, maxfun, xatol, disp)
 
     result = Result(fun=fval, status=flag, success=(flag == 0),
                     message={0: 'Solution found.',
@@ -2755,6 +2755,20 @@ def show_options(solver, method=None):
     * trust-ncg options:
         see dogleg options.
 
+    ** minimize_scalar options
+
+    * brent options:
+        xtol : float
+            Relative error in solution `xopt` acceptable for convergence.
+
+    * bounded options:
+        xatol : float
+            Absolute error in solution `xopt` acceptable for convergence.
+
+    * golden options:
+        xtol : float
+            Relative error in solution `xopt` acceptable for convergence.
+
     ** root options
 
     * hybrd options:


Test patch: