Repository: scipy/scipy

Issue:
optimize.differential_evolution stopping criterion has unintuitive dependence on objective
I've been using optimize.differential_evolution with considerable success but I've noticed a quirk which can greatly perturb the number of iterations and error.  As an example, the following code finds the minimum of the Rosenbrook function (optimize.rosen) offset by a constant using optimize.differential_evolution.  It evaluates each optimization problem multiple times, then prints a table of the offset value, the average number of iterations, and the average error (the squared error on the true minima, at [1, 1]).

```
from __future__ import division

import numpy as np

from scipy.optimize import differential_evolution, rosen

N_VARS = 2
VAR_BOUND = (-5, 5)
REPLICATES = 100

bounds = [VAR_BOUND] * N_VARS

print 'offset\t<iters>\t<error>'

for offset in (0, 1, 10, 100, -10):
    total_iters = 0
    total_error = 0

    for i in xrange(REPLICATES):
        result = differential_evolution(
            lambda x: rosen(x) + offset,
            bounds
            )

        total_iters += result.nit
        total_error += np.sum(np.square(result.x - 1))

    print '{:5n}\t{:6.1f}\t{:0.0e}'.format(
        offset,
        total_iters / REPLICATES,
        total_error / REPLICATES
        )
```

Output:

```
offset  <iters> <error>
    0     75.0  1e-19
    1     20.2  7e-08
   10     16.5  3e-06
  100     12.6  2e-05
  -10     16.8  8e-07
```

The trend should be obvious - the larger the offset, the fewer the number of iterations required to solve.  Likewise, the final error increases by orders of magnitude, though it is still relatively small.  This issue can be tracked to the stopping criterion, which terminates when

```
EPS = np.finfo(np.float64).eps
std(pop)/(mean(pop)+EPS) < tol
```

where "pop" is the vector of the population's objective scores, and "tol" is a parameter supplied to differential_evolution (default is 0.1).  Since the Rosenbrook function has a minimal value of zero, the stopping criterion blows up as the population approaches the minima, only slowing down once the "EPS" term dominates the denominator.

This stopping criterion might make sense if 1) you know for a fact that the minima is zero and 2) you need extremely high precision on your result.  This might make sense for test problems, like the Rosenbrook function.  In general, it's unintuitive and undocumented.

I have two suggestions.  First, replace the current stopping criterion with a simpler function that is insensitive to the population mean.  There is a whole host of possible criterion (see http://link.springer.com/chapter/10.1007%2F978-3-540-68830-3_4#page-1) - I think the standard deviation alone is an excellent default.  Second, allow the user to optionally pass any stopping criterion function they want.

Edit: deleted extraneous code



Discussion:
Yeah, I can see how it's going to take much longer to converge for cases where the minimum is zero.  The scaling by the mean of the population energies is done to normalise the standard deviation.
There will be problems where the population energies will be high and problems where the population energies will be low. The corresponding standard deviations for those problems will be of different magnitudes. So I don't think it's enough just to use standard deviation alone, you have to have an idea of the spread of the population: are they tightly bunched (in which case you want to stop), or loosely bunched (you want to carry on)? Dividing by standard deviation by the mean is one way of seeing if this occurs. Unfortunately if the global minimum is zero then there is going to be this problem. I use `differential_evolution` for curvefitting, so the global minimum is always much larger.
Adding a stopping criterion function is one way to go, perhaps the `callback` function can be modified to accept the population energy array as a kwd.
I can't access the article you linked to because it's paywalled. What solutions do they suggest?

Thanks Andy, sorry about the paywall.  There are three general categories:
- improvement-based: stop after failing to find a (significant) improvement for N iterations
- movement-based: stop if the average population movement (in parameter- or objective-space) if below some threshold
- distribution-based: stop based on some property of the population at a single iterate

The latter has many variants:
- stop if the largest distance between the best and all other population members falls below some threshold (parameter- or objective-space)
- stop if the standard deviation of the population falls below some threshold

They go on to suggest that max distance between the best and all other population members, in objective-space, was the most promising stopping-criterion.  This would be pretty easy to implement.  They claim that choosing the threshold is easy and recommend one order of magnitude less than the desired error.

I much prefer distribution based termination. Given the stochastic nature of the algorithm I think that improvement and movement based criteria have the possibility of terminating too early.
I agree that the max distance or standard deviation of the population members is a good way to go for distribution based. However, as I said in my earlier post they need to be normalised for the magnitude of the population energy. Remember that the `differential_evolution` function needs to work out of the box for a wide range of problems. 
For example one might be doing a least squares analysis, with the objective function returning `chi**2`. The best solution `chi**2` might be 1 or it might be 1000. If the criterion is standard deviation and the standard deviation has to be less than 0.05 to terminate, then when the best solution `chi**2` is 1 you will get convergence when the population has just converged on the answer (68 % of the population will lie in between 0.95 and 1.05). However, if the best solution `chi**2` is 1000 then the population will probably iterate for a lot longer because 68 % of the population will need to have an energy between 999.95 and 1000.05 (it's also going to depend on the energy landscape around the minimum). In the former example sd/mean is 5 %, in the latter example it is 0.005 %. Standard deviation or largest distance only indicate how tightly these distributions are dispersed if the central tendency (mean or median) is known.

Unfortunately when the best solution is 0 then one will get problems as the mean tends to zero. I'm not sure how frequently real life problems experience this.

At this point in time I think that changing the convergence behaviour is going to lead to problems with existing third party code. A reasonable suggestion is to allow the `tol` kwd to accept a callable, as you suggested. That function allows a user defined termination.

Please note that in 0.18 the `DifferentialEvolutionSolver` class will be iterable, which allows you to evolve the population an iteration at a time. After each iteration you will be able to test for convergence with your own metric.

```
from scipy.optimize._differentialevolution import DifferentialEvolutionSolver
from scipy.optimize import rosen
import numpy as np
bounds = [(-2, 2), (-2, 2)]
solver = DifferentialEvolutionSolver(rosen, bounds)
while True:
    best_x, best_energy = next(solver)
    # define your own convergence criterion.
    # best_energy = solver.population_energies[0] = np.min(solver.population_energies)
    if np.max(solver.population_energies) - np.min(solver.population_energies) < 0.01:
        break
```

You mentioned in your post:

> that choosing the threshold is easy and recommend one order of magnitude less than the desired error.

If I understand correctly, in the least squares example I gave above choosing the threshold would not be possible. Before the minimisation starts the global energy minimum is not known, so how can one choose a threshold?

Differential equation solvers (among many other numerical algorithms) face
a similar problem: when the user provides an error limit to aim for, do
they want relative or absolute error? Relative error helps deal with
situations where the quantities are not of order unity - if you're
modelling the motions of solar system objects in meters, then using 1e-8 as
your tolerance is going to be unnecessarily challenging. On the other hand,
if your quantities really are of order unity but one value is very close to
zero, then it may not be helpful to sweat over computing that one quantity
to much tighter tolerances. So it's normal to use both a relative tolerance
and an absolute tolerance, provided by the user, and stop only when the sum
in quadrature of the estimated relative and absolute errors is small enough.

How to provide these error estimates is another question, depending on the
workings of differential evolution. But the unintuitive scaling you're
seeing comes from a relative definition of "small enough".

Anne

On Mon, Jul 18, 2016 at 2:57 AM Andrew Nelson notifications@github.com
wrote:

> I much prefer distribution based termination. Given the stochastic nature
> of the algorithm I think that improvement and movement based criteria have
> the possibility of terminating too early.
> I agree that the max distance or standard deviation of the population
> members is a good way to go for distribution based. However, as I said in
> my earlier post they need to be normalised for the magnitude of the
> population energy. Remember that the differential_evolution function
> needs to work out of the box for a wide range of problems.
> For example one might be doing a least squares analysis, with the
> objective function returning chi**2. The best solution chi**2 might be 1
> or it might be 1000. If the criterion is standard deviation and the
> standard deviation has to be less than 0.05 to terminate, then when the
> best solution chi**2 is 1 you will get convergence when the population
> has just converged on the answer (68 % of the population will lie in
> between 0.95 and 1.05). However, if the best solution chi**2 is 1000 then
> the population will probably iterate for a lot longer because 68 % of the
> population will need to have an energy between 999.95 and 1000.05 (it's
> also going to depend on the energy landscape around the minimum). In the
> former example sd/mean is 5 %, in the latter example it is 0.005 %.
> Standard deviation or largest distance only indicate how tightly these
> distributions are dispersed if the central tendency (mean or median) is
> known.
> 
> Unfortunately when the best solution is 0 then one will get problems as
> the mean tends to zero. I'm not sure how frequently real life problems
> experience this.
> 
> At this point in time I think that changing the convergence behaviour is
> going to lead to problems with existing third party code. A reasonable
> suggestion is to allow the tol kwd to accept a callable, as you
> suggested. That function allows a user defined termination.
> 
> Please note that in 0.18 the DifferentialEvolutionSolver class will be a
> generator, which allows you to evolve the population an iteration at a
> time. After each iteration you will be able to test for convergence with
> your own metric.
> 
> from scipy.optimize._differentialevolution import DifferentialEvolutionSolver
> from scipy.optimize import rosen
> import numpy as np
> bounds = [(-2, 2), (-2, 2)]
> solver = DifferentialEvolutionSolver(rosen, bounds)
> while True:
>     best_x, best_energy = next(solver)
>     # define your own convergence criterion.
>     # best_energy = solver.population_energies[0] = np.min(solver.population_energies)
>     if np.max(solver.population_energies) - np.min(solver.population_energies) < 0.01:
>         break
> 
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> https://github.com/scipy/scipy/issues/6382#issuecomment-233214398, or mute
> the thread
> https://github.com/notifications/unsubscribe-auth/AAIWowaAtWWKH0K0V1LichnZ9k9jNlswks5qWs-UgaJpZM4JN2Qp
> .

Regarding the chi**2 example - I don't follow this very well, but I'd say I'm unconvinced that anything other than the energy landscape determines the rate of convergence.  Perhaps in curve-fitting, you specifically want less precision on poor fits.  The changes in 0.18 sound nice, but it sounds like for my purposes I'll need to write my own solver.  I need control on precision over a range of objective values.

@aarchiba That's not very relevant.  Think of it more like this.  I'm modeling the movement of an object in 1D, and I want my differential equation solver to run until it stops, i.e. dx/dt is very near zero.  Would you divide the velocity by the position to get a "normalized" velocity, and then use that as the stopping criterion?  Of course not - the behavior would be totally nonsensical barring a very specific problem definition.


Resolving patch:
diff --git a/scipy/optimize/_differentialevolution.py b/scipy/optimize/_differentialevolution.py
--- a/scipy/optimize/_differentialevolution.py
+++ b/scipy/optimize/_differentialevolution.py
@@ -17,7 +17,7 @@ def differential_evolution(func, bounds, args=(), strategy='best1bin',
                            maxiter=1000, popsize=15, tol=0.01,
                            mutation=(0.5, 1), recombination=0.7, seed=None,
                            callback=None, disp=False, polish=True,
-                           init='latinhypercube'):
+                           init='latinhypercube', atol=0):
     """Finds the global minimum of a multivariate function.
     Differential Evolution is stochastic in nature (does not use gradient
     methods) to find the minimium, and can search large areas of candidate
@@ -64,10 +64,10 @@ def differential_evolution(func, bounds, args=(), strategy='best1bin',
         A multiplier for setting the total population size.  The population has
         ``popsize * len(x)`` individuals.
     tol : float, optional
-        When the mean of the population energies, multiplied by tol,
-        divided by the standard deviation of the population energies
-        is greater than 1 the solving process terminates:
-        ``convergence = mean(pop) * tol / stdev(pop) > 1``
+        Relative tolerance for convergence, the solving stops when
+        ``np.std(pop) <= atol + tol * np.abs(np.mean(population_energies))``,
+        where and `atol` and `tol` are the absolute and relative tolerance
+        respectively.
     mutation : float or tuple(float, float), optional
         The mutation constant. In the literature this is also known as
         differential weight, being denoted by F.
@@ -114,6 +114,11 @@ def differential_evolution(func, bounds, args=(), strategy='best1bin',
         maximize coverage of the available parameter space. 'random' initializes
         the population randomly - this has the drawback that clustering can
         occur, preventing the whole of parameter space being covered.
+    atol : float, optional
+        Absolute tolerance for convergence, the solving stops when
+        ``np.std(pop) <= atol + tol * np.abs(np.mean(population_energies))``,
+        where and `atol` and `tol` are the absolute and relative tolerance
+        respectively.
 
     Returns
     -------
@@ -201,8 +206,7 @@ def differential_evolution(func, bounds, args=(), strategy='best1bin',
                                          recombination=recombination,
                                          seed=seed, polish=polish,
                                          callback=callback,
-                                         disp=disp,
-                                         init=init)
+                                         disp=disp, init=init, atol=atol)
     return solver.solve()
 
 
@@ -249,10 +253,10 @@ class DifferentialEvolutionSolver(object):
         A multiplier for setting the total population size.  The population has
         ``popsize * len(x)`` individuals.
     tol : float, optional
-        When the mean of the population energies, multiplied by tol,
-        divided by the standard deviation of the population energies
-        is greater than 1 the solving process terminates:
-        ``convergence = mean(pop) * tol / stdev(pop) > 1``
+        Relative tolerance for convergence, the solving stops when
+        ``np.std(pop) <= atol + tol * np.abs(np.mean(population_energies))``,
+        where and `atol` and `tol` are the absolute and relative tolerance
+        respectively.
     mutation : float or tuple(float, float), optional
         The mutation constant. In the literature this is also known as
         differential weight, being denoted by F.
@@ -298,6 +302,11 @@ class DifferentialEvolutionSolver(object):
 
             - 'latinhypercube'
             - 'random'
+    atol : float, optional
+        Absolute tolerance for convergence, the solving stops when
+        ``np.std(pop) <= atol + tol * np.abs(np.mean(population_energies))``,
+        where and `atol` and `tol` are the absolute and relative tolerance
+        respectively.
     """
 
     # Dispatch of mutation strategy method (binomial or exponential).
@@ -316,7 +325,7 @@ def __init__(self, func, bounds, args=(),
                  strategy='best1bin', maxiter=1000, popsize=15,
                  tol=0.01, mutation=(0.5, 1), recombination=0.7, seed=None,
                  maxfun=np.inf, callback=None, disp=False, polish=True,
-                 init='latinhypercube'):
+                 init='latinhypercube', atol=0):
 
         if strategy in self._binomial:
             self.mutation_func = getattr(self, self._binomial[strategy])
@@ -328,7 +337,9 @@ def __init__(self, func, bounds, args=(),
 
         self.callback = callback
         self.polish = polish
-        self.tol = tol
+
+        # relative and absolute tolerances for convergence
+        self.tol, self.atol = tol, atol
 
         # Mutation constant should be in [0, 2). If specified as a sequence
         # then dithering is performed.
@@ -511,8 +522,7 @@ def solve(self):
                       % (nit,
                          self.population_energies[0]))
 
-            # stop when the fractional s.d. of the population is less than tol
-            # of the mean energy
+            # should the solver terminate?
             convergence = self.convergence
 
             if (self.callback and
@@ -524,7 +534,10 @@ def solve(self):
                                   'by returning True')
                 break
 
-            if convergence < self.tol or warning_flag:
+            intol = (np.std(self.population_energies) <=
+                     self.atol +
+                     self.tol * np.abs(np.mean(self.population_energies)))
+            if warning_flag or intol:
                 break
 
         else:


Test patch: