Repository: scipy/scipy

Issue:
Bug in Differential Evolution (Array-length with floating point rounding error)
Hello,

Thanks for making scipy!

There is a bug in scipy 0.15.1 in the file _differentialevolution.py

The bug is in the function init_population_lhs(self) in this line:

```
rdrange += np.atleast_2d(np.arange(0., 1., segsize)).T
```

and specifically in this part:

```
np.arange(0., 1., segsize)
```

The problem is that floating-point rounding errors can sometimes create arrays of the wrong length. For example, if the dimensionality of the search-space is 29 and the population size is 35 then the bug is triggered.

I have fixed the source-code and attached it below. I have also revised and cleaned up the code a bit. The commenting was also confusing and misleading. I'm not sure mine is much better because I haven't studied exactly what the function is supposed to do (again, the commenting was lacking).

There are still some things in the function that should be improved, but this would require changes to e.g. the class init as well. I have mentioned these issues in the comments.

I have also noticed other strange things in other parts of scipy's DE source-code so I can imagine there are more bugs.

DE is a very useful optimizer and many people probably appreciate that it is now included in scipy, but the implementation needs to be improved. I would suggest fresh and critical eyes from the scipy team go over the source-code to simplify, test and document it properly. I would have liked to help you with this but I'm new to Python.

I have attached my own DE implementation for Matlab as a reference so you can see how simple it can be done.

Furthermore, please use the typical DE terminology. You have renamed the DE control parameters to 'mutation' and 'recombination'. Please use their typical names in the research literature 'CR' and 'F' so a practitioner can easily see which is which, or use the full names such as crossover_probability (instead of CR) and differential_weight (instead of F).

Not everything is bad with scipy's DE implementation and there are some pretty nice features! I particularly like the convergence detection system and that L-BFGS-B optimization can be continued from DE's best found solution - assuming it isn't buggy! :-)

Thanks again for making scipy!

Cheers, Magnus.
###### 
# _differentialevolution.py
# OLD Version:

```
def init_population_lhs(self):
    """
    Initializes the population with Latin Hypercube Sampling
    Latin Hypercube Sampling ensures that the sampling of parameter space
    is maximised.
    """
    samples = np.size(self.population, 0)
    N = np.size(self.population, 1)
    rng = self.random_number_generator

    # Generate the intervals
    segsize = 1.0 / samples

    # Fill points uniformly in each interval
    rdrange = rng.rand(samples, N) * segsize
    rdrange += np.atleast_2d(np.arange(0., 1., segsize)).T

    # Make the random pairings
    self.population = np.zeros_like(rdrange)

    for j in range(N):
        order = rng.permutation(range(samples))
        self.population[:, j] = rdrange[order, j]
```
###### 
# _differentialevolution.py
# NEW version:

```
def init_population_lhs(self):
    """
    Initializes the population with Latin Hypercube Sampling
    Latin Hypercube Sampling ensures that the sampling of parameter space
    is maximised.

    NOTE: This should describe with a math formula what is being calculated!
    """

    # Get number of samples for each candidate solution.
    # This is messy and should be self.num_samples instead.
    num_samples = np.size(self.population, 0)

    # Get search-space dimensionality.
    # This is messy and should be self.dim instead.
    dim = np.size(self.population, 1)

    # Get RNG.
    rng = self.random_number_generator

    # Generate random samples.
    samples = rng.rand(num_samples, dim)

    # Scale used for random samples.
    scale = 1.0 / num_samples

    # Scale the samples.
    samples *= scale

    # Offset each individual sample so they are in different ranges.
    # This spreads out the samples evenly.
    offset = scale * np.arange(1, num_samples + 1)
    samples += np.atleast_2d(offset).T

    # Allocate array for population of candidate solutions and initialize to zero.
    self.population = np.zeros_like(samples)

    # Initialize population of candidate solutions by permutation of the random samples.
    for j in range(dim):
        index = rng.permutation(range(num_samples))
        self.population[:, j] = samples[index, j]
```
###### 

% ------------------------------------------------------
% SwarmOps - Heuristic optimization for Matlab
% Copyright (C) 2003-2010 Magnus Erik Hvass Pedersen.
% Please see the file license.txt for license details.
% SwarmOps on the internet: http://www.Hvass-Labs.org/
% ------------------------------------------------------

% Differential Evolution (DE) by Storner and Price (1).
% DE is an optimization method that does not use the
% gradient of the problem. This is the DE/rand/1/bin variant.
% Literature references:
% (1) R. Storn and K. Price. Differential evolution - a simple
%     and efficient heuristic for global optimization over
%     continuous spaces. Journal of Global Optimization,
%     11:341-359, 1997.
% Parameters:
%     problem; name or handle of optimization problem, e.g. @myproblem.
%     data; data-struct, see e.g. the file myproblemdata.m
%     parameters; behavioural parameters for optimizer,
%                 see file deparameters.m
% Returns:
%     bestX; best found position in the search-space.
%     bestFitness; fitness of bestX.
%     evaluations; number of fitness evaluations performed.

function [bestX, bestFitness, evaluations] = de(problem, data, parameters)

```
% Copy data contents to local variables for convenience.
n = data.Dim;
acceptableFitness = data.AcceptableFitness;
maxEvaluations = data.MaxEvaluations;
lowerBound = data.LowerBound;
upperBound = data.UpperBound;

% Behavioural parameters for this optimizer.
np = parameters(1);       % Population size
cr = parameters(2);       % Crossover probability.
f = parameters(3);        % Differential weight.

% Initialize population.
x = initpopulation(np, n, data.LowerInit, data.UpperInit);

% Compute fitness for all agents in population.
fitness = zeros(1, np); % Pre-allocate for efficiency.
for i=1:np
    fitness(i) = feval(problem, x(i,:), data);
end

% Determine fitness and index of best agent.
[bestFitness, bestIndex] = min(fitness);

% Perform optimization iterations until acceptable fitness
% is achieved or the maximum number of fitness evaluations
% has been performed.
evaluations = np; % Fitness evaluations above count as iterations.
while (evaluations < maxEvaluations) && (bestFitness > acceptableFitness)

    % Pick random and distinct agents from population.
    indices = randperm(np);
    i = indices(1);        % Agent to update.
    a = indices(2);        % Other agent a.
    b = indices(3);        % Other agent b.
    c = indices(4);        % Other agent c.

    % Pick random index.
    R = randindex(n);

    % Copy old agent as a basis for the new agent.
    y = x(i,:);

    % Compute new agent using DE formula.
    for j=1:n
        if (j==R) || (rand(1,1) < cr)
            y(j) = x(a,j) + f * (x(b,j) - x(c,j));
        end
    end

    % Bound position to search-space.
    y = bound(y, lowerBound, upperBound);

    % Compute fitness.
    newFitness = feval(problem, y, data);

    % Update agent if fitness improvement.
    if newFitness < fitness(i)
        % Update fitness.
        fitness(i) = newFitness;

        % Update agent position.
        x(i,:) = y;

        % Update population's best, if improvement.
        if newFitness < bestFitness
            bestFitness = newFitness;
            bestIndex = i;
        end
    end

    % Increment counter.
    evaluations = evaluations + 1;
end

% Set best found solution (bestFitness already set).
bestX = x(bestIndex,:);
```

end

% ------------------------------------------------------



Discussion:
The arange issue seems to have been fixed in 77e7c91ac

I haven't been involved with this code, but it could be helpful if you list the "strange things" you encountered.
cc @andyfaff 

The bug that was mentioned was fixed in 77e7c91.

Wikipedia has an article on [Latin Hypercube Sampling](https://en.wikipedia.org/wiki/Latin_hypercube_sampling). The relevant part is: 

> In the context of statistical sampling, a square grid containing sample positions is a Latin square if (and only if) there is only one sample in each row and each column. A Latin hypercube is the generalisation of this concept to an arbitrary number of dimensions, whereby each sample is the only one in each axis-aligned hyperplane containing it.
> When sampling a function of N variables, the range of each variable is divided into M equally probable intervals. M sample points are then placed to satisfy the Latin hypercube requirements; note that this forces the number of divisions, M, to be equal for each variable.

The docstring could certainly be updated with this information.

I'm not sure what to do about the DE terminology.  I started using the terminology that's used in the paper by [Wormington](http://rsta.royalsocietypublishing.org/content/357/1761/2827.short).  Changes could be made internally, and in the documentation, referring to `crossover_probability` and `mutation_weight`.  However, the code has been released for over a year and I'm not sure that the function signature should be changed, it would break peoples code.  We could deprecate and change the keywords, but I'm not sure it's worth it.

> noticed other strange things in other parts of scipy's DE source-code

Please enumerate the 'strange things'.  If there are bugs then they should certainly be fixed.

> go over the source-code to simplify, test and document it properly.

The differential_evolution code has a wide range of unit tests.  Any extra tests that increase coverage are always welcomed.

When the original version was written a large amount of work went into polishing it (clarity, simplification, testing).  Any suggestions and PR's as to how further simplification can be achieved are welcome.  Don't forget that the implementation has a wide range of mutation strategies, which does increase the SLOC.  Extra code is also required to make the output similar to that of the other minimizers.

I had a look at your matlab code and the general algorithm in the two is very similar for rand/1/bin.  However, I did have one comment.  Normally the entire population is iterated through in sequence. The original Storn and Price paper uses sequential iteration of the population, as does the Wormington paper I referred to above. The _algorithm_ section of the Wikipedia page does as well.  However, I think the _sample code_ section visits the population in a random manner.

 I'm not sure that your code is performing sequential iteration of the population. You have:

```
while (evaluations < maxEvaluations) && (bestFitness > acceptableFitness)
    % Pick random and distinct agents from population.
    indices = randperm(np);
    i = indices(1);        % Agent to update.
    a = indices(2);        % Other agent a.
    b = indices(3);        % Other agent b.
    c = indices(4);        % Other agent c.
```

`i` should be calculated as something like `i = evaluations % np`, which would correspond to the entire population being iterated in sequence.  The `randperm` function could well be doing this, but it seems like  `i` is also a random number. If it is then random entries of the population are being mutated on each evaluation, which I don't think is correct.

Thanks for the quick reply and sorry for my delay.

I have never seen the Wormington paper before. Perhaps that is the source of your unusual terminology.

I appreciate that you don't want to make changes that would break the API for other users. On the other hand it is very confusing that you call the parameters mutation and recombination. Scipy is also only in v. 0.16 or so (i.e. under development), so I think it would be alright to make large changes to the API.

At the very least the parameter docstrings could be like this:

```
mutation : float or tuple(float, float), optional
    The mutation constant aka. differential weight and denoted F in the literature.

recombination : float, optional
    The recombination constant, should be in the range [0, 1]. Aka. crossover probability and denoted CR in the literature.
```

I also saw someone else on the forum mention that 'maxiter' and 'popsize' are confusing and I agree. popsize is supposed to be the size of the population - you should not scale this with the dimensionality of the search-space. And maxiter should match the meaning of scipy - I suppose it is the max number of function evaluations for the problem being optimized? but you scale maxiter with popsize and search-space dimensionality.

What I meant with "strange things" in the code was exemplified in the function I had attached and modified to show how I think it should be done. I changed variable names and added comments to try and clarify what the function was doing. I also noted a few things were messy in the function and required changes to the class init. Your code has practically no comments, and the few comments are often misplaced and / or misleading. Your implementation is quite complex because it has so many features, few comments and because of your personal coding style. So it is hard to understand and maintain the code for others and there's likely more bugs in it.

Unit tests are great - but obviously insufficient and cannot substitute for quality code with good comments.

A suggestion is that you remove most of the DE variants so you can streamline the code much more. My understanding is that scipy should have a broad range of high quality and thoroughly tested algorithms, rather than numerous minor variants of the same algorithm that makes the implementation complex and error-prone.

There are probably hundreds if not thousands of DE variants and the ones you include are not even those considered state-of-the-art by the research community. However, there is no real scientific evidence that even the state-of-the-art variants have superior performance, see my thesis which has detailed experiments on this subject:

http://www.hvass-labs.org/people/magnus/thesis/pedersen08thesis.pdf

My SwarmOps library for C# and Java also has several DE variants because I needed them for my research. It was a pain to program and although I put effort into keeping the code clean and well-documented it also got messy because of the many DE variants.

When I look at my C# source-code now, I mainly read the comments rather than the actual code-statements, see e.g. DESuite.cs (contains some of the variants you also have) and JDE.cs (JDE was supposed to be the best DE variant at the time - but of course didn't hold up under scrutiny).

SwarmOps has been forked on github:

https://github.com/firestrand/SwarmOps/blob/master/SwarmOps/Optimizers/DESuite.cs

https://github.com/firestrand/SwarmOps/blob/master/SwarmOps/Optimizers/JDE.cs

Reading just the comments in my C# code I quickly get an understanding of what the code is doing and how it flows, without having to understand the actual code. In a few minutes I can understand the code even though I wrote it maybe 7 years ago and haven't worked in this field for years.

But I can spend a long time reading scipy's DE code and I still don't understand it. If a bug occurs I might be able to fix that single line (as I did in the example above), but it's difficult for me to understand the context of that line of code so I might introduce other bugs.

My suggestion is again that you simplify scipy's DE implementation and improve the coding style and commenting. Ideally, I think scipy should just have one DE variant akin to the Matlab-code I previously attached.

Also take a look at PyEC which is implemented by a skilled researcher in evolutionary computation who is now working at a leading AI research lab. PyEC also just has one basic DE implementation (although it unfortunately also lacks comments): 

https://pypi.python.org/pypi/PyEC

I've looked at other parts of scipy.optimize and the coding style generally uses cryptic variable names, poor control-flows (e.g. lots of breaks inside while-loops), poor code structuring (e.g. related statements are not grouped together properly), and almost no comments! It is very difficult for others to understand and maintain such code. I wonder if the rest of scipy is programmed in the same style as scipy.optimize and what this might mean in terms of hidden bugs.

Scipy is very popular so I think it would be worthwhile and wise for the scipy community to consider requirements for the coding style in general. Ask yourself if the code can be easily understood and maintained by others 10 years from now?

I hope this makes sense and I'm not coming across as a troll :-)


Resolving patch:
diff --git a/scipy/optimize/_differentialevolution.py b/scipy/optimize/_differentialevolution.py
--- a/scipy/optimize/_differentialevolution.py
+++ b/scipy/optimize/_differentialevolution.py
@@ -323,8 +323,8 @@ def __init__(self, func, bounds, args=(),
         self.polish = polish
         self.tol = tol
 
-        #Mutation constant should be in [0, 2). If specified as a sequence
-        #then dithering is performed.
+        # Mutation constant should be in [0, 2). If specified as a sequence
+        # then dithering is performed.
         self.scale = mutation
         if (not np.all(np.isfinite(mutation)) or
                 np.any(np.array(mutation) >= 2) or
@@ -364,13 +364,17 @@ def __init__(self, func, bounds, args=(),
         self.__scale_arg1 = 0.5 * (self.limits[0] + self.limits[1])
         self.__scale_arg2 = np.fabs(self.limits[0] - self.limits[1])
 
-        parameter_count = np.size(self.limits, 1)
+        self.parameter_count = np.size(self.limits, 1)
+
         self.random_number_generator = _make_random_gen(seed)
 
-        #default initialization is a latin hypercube design, but there
-        #are other population initializations possible.
-        self.population = np.zeros((popsize * parameter_count,
-                                    parameter_count))
+        # default population initialization is a latin hypercube design, but
+        # there are other population initializations possible.
+        self.num_population_members = popsize * self.parameter_count
+
+        self.population_shape = (self.num_population_members,
+                                 self.parameter_count)
+
         if init == 'latinhypercube':
             self.init_population_lhs()
         elif init == 'random':
@@ -379,35 +383,41 @@ def __init__(self, func, bounds, args=(),
             raise ValueError("The population initialization method must be one"
                              "of 'latinhypercube' or 'random'")
 
-        self.population_energies = np.ones(
-            popsize * parameter_count) * np.inf
+        self.population_energies = (np.ones(self.num_population_members)
+                                    * np.inf)
 
         self.disp = disp
 
     def init_population_lhs(self):
         """
-        Initializes the population with Latin Hypercube Sampling
-        Latin Hypercube Sampling ensures that the sampling of parameter space
-        is maximised.
+        Initializes the population with Latin Hypercube Sampling.
+        Latin Hypercube Sampling ensures that each parameter is uniformly
+        sampled over its range.
         """
-        samples = np.size(self.population, 0)
-        N = np.size(self.population, 1)
         rng = self.random_number_generator
 
-        # Generate the intervals
-        segsize = 1.0 / samples
+        # Each parameter range needs to be sampled uniformly. The scaled
+        # parameter range ([0, 1)) needs to be split into
+        # `self.num_population_members` segments, each of which has the following
+        # size:
+        segsize = 1.0 / self.num_population_members
 
-        # Fill points uniformly in each interval
-        rdrange = rng.rand(samples, N) * segsize
-        rdrange += np.atleast_2d(
-            np.linspace(0., 1., samples, endpoint=False)).T
+        # Within each segment we sample from a uniform random distribution.
+        # We need to do this sampling for each parameter.
+        samples = (segsize * rng.random_sample(self.population_shape)
 
-        # Make the random pairings
-        self.population = np.zeros_like(rdrange)
+        # Offset each segment to cover the entire parameter range [0, 1)
+                   + np.linspace(0., 1., self.num_population_members,
+                                 endpoint=False)[:, np.newaxis])
 
-        for j in range(N):
-            order = rng.permutation(range(samples))
-            self.population[:, j] = rdrange[order, j]
+        # Create an array for population of candidate solutions.
+        self.population = np.zeros_like(samples)
+
+        # Initialize population of candidate solutions by permutation of the
+        # random samples.
+        for j in range(self.parameter_count):
+            order = rng.permutation(range(self.num_population_members))
+            self.population[:, j] = samples[order, j]
 
     def init_population_random(self):
         """
@@ -415,7 +425,7 @@ def init_population_random(self):
         can possess clustering, Latin Hypercube sampling is generally better.
         """
         rng = self.random_number_generator
-        self.population = rng.random_sample(self.population.shape)
+        self.population = rng.random_sample(self.population_shape)
 
     @property
     def x(self):
@@ -483,23 +493,34 @@ def solve(self):
             if self.dither is not None:
                 self.scale = self.random_number_generator.rand(
                 ) * (self.dither[1] - self.dither[0]) + self.dither[0]
-            for candidate in range(np.size(self.population, 0)):
+
+            for candidate in range(self.num_population_members):
                 if nfev > self.maxfun:
                     warning_flag = True
                     status_message = _status_message['maxfev']
                     break
 
+                # create a trial solution
                 trial = self._mutate(candidate)
+
+                # ensuring that it's in the range [0, 1)
                 self._ensure_constraint(trial)
+
+                # scale from [0, 1) to the actual parameter value
                 parameters = self._scale_parameters(trial)
 
+                # determine the energy of the objective function
                 energy = self.func(parameters, *self.args)
                 nfev += 1
 
+                # if the energy of the trial candidate is lower than the
+                # original population member then replace it
                 if energy < self.population_energies[candidate]:
                     self.population[candidate] = trial
                     self.population_energies[candidate] = energy
 
+                    # if the trial candidate also has a lower energy than the
+                    # best solution then replace that as well
                     if energy < self.population_energies[0]:
                         self.population_energies[0] = energy
                         self.population[0] = trial
@@ -584,9 +605,10 @@ def _mutate(self, candidate):
         create a trial vector based on a mutation strategy
         """
         trial = np.copy(self.population[candidate])
-        parameter_count = np.size(trial, 0)
 
-        fill_point = self.random_number_generator.randint(0, parameter_count)
+        rng = self.random_number_generator
+
+        fill_point = rng.randint(0, self.parameter_count)
 
         if (self.strategy == 'randtobest1exp'
                 or self.strategy == 'randtobest1bin'):
@@ -596,7 +618,7 @@ def _mutate(self, candidate):
             bprime = self.mutation_func(self._select_samples(candidate, 5))
 
         if self.strategy in self._binomial:
-            crossovers = self.random_number_generator.rand(parameter_count)
+            crossovers = rng.rand(self.parameter_count)
             crossovers = crossovers < self.cross_over_probability
             # the last one is always from the bprime vector for binomial
             # If you fill in modulo with a loop you have to set the last one to
@@ -608,12 +630,11 @@ def _mutate(self, candidate):
 
         elif self.strategy in self._exponential:
             i = 0
-            while (i < parameter_count and
-                   self.random_number_generator.rand() <
-                   self.cross_over_probability):
+            while (i < self.parameter_count and
+                   rng.rand() < self.cross_over_probability):
 
                 trial[fill_point] = bprime[fill_point]
-                fill_point = (fill_point + 1) % parameter_count
+                fill_point = (fill_point + 1) % self.parameter_count
                 i += 1
 
             return trial
@@ -651,8 +672,8 @@ def _best2(self, samples):
         """
         r0, r1, r2, r3 = samples[:4]
         bprime = (self.population[0] + self.scale *
-                            (self.population[r0] + self.population[r1]
-                           - self.population[r2] - self.population[r3]))
+                  (self.population[r0] + self.population[r1]
+                   - self.population[r2] - self.population[r3]))
 
         return bprime
 
@@ -662,17 +683,17 @@ def _rand2(self, samples):
         """
         r0, r1, r2, r3, r4 = samples
         bprime = (self.population[r0] + self.scale *
-                 (self.population[r1] + self.population[r2] -
-                  self.population[r3] - self.population[r4]))
+                  (self.population[r1] + self.population[r2] -
+                   self.population[r3] - self.population[r4]))
 
         return bprime
 
     def _select_samples(self, candidate, number_samples):
         """
-        obtain random integers from range(np.size(self.population, 0)),
+        obtain random integers from range(self.num_population_members),
         without replacement.  You can't have the original candidate either.
         """
-        idxs = list(range(np.size(self.population, 0)))
+        idxs = list(range(self.num_population_members))
         idxs.remove(candidate)
         self.random_number_generator.shuffle(idxs)
         idxs = idxs[:number_samples]


Test patch: