Analyzing Adaptive Parameter Landscapes in Parameter
Adaptation Methods for Differential Evolution
Ryoji Tanabe
Yokohama National University
Yokohama, Japanrt.ryoji.tanabe@gmail.com

ABSTRACT
Since the scale factor and the crossover rate significantly influence
the performance of differential evolution (DE), parameter adaptation methods (PAMs) for the two parameters have been well studied
in the DE community. Although PAMs can sufficiently improve theeffectiveness of DE, PAMs are poorly understood (e.g., the working
principle of PAMs). One of the difficulties in understanding PAMs
comes from the unclarity of the parameter space that consists of thescale factor and the crossover rate. This paper addresses this issue
by analyzing adaptive parameter landscapes in PAMs for DE. First,
we propose a concept of an adaptive parameter landscape, which
captures a moment in a parameter adaptation process. For each iteration, each individual in the population has its adaptive parameter
landscape. Second, we propose a method of analyzing adaptive parameter landscapes using a 1-step-lookahead greedy improvement
metric. Third, we examine adaptive parameter landscapes in three
PAMs by using the proposed method. Results provide insightful
information about PAMs in DE.

CCS CONCEPTS• Mathematics of computing → Evolutionary algorithms;

KEYWORDS
DE, parameter adaptation methods, landscape analysis
ACM Reference Format:
Ryoji Tanabe. 2020. Analyzing Adaptive Parameter Landscapes in Parameter
Adaptation Methods for Differential Evolution. In Genetic and Evolutionary
Computation Conference (GECCO ’20), July 8–12, 2020, Cancún, Mexico. ACM,
New York, NY, USA, 9 pages. https://doi.org/10.1145/3377930.3389820

1

INTRODUCTION

This paper considers a black-box numerical optimization. These
problems involve finding a d-dimensional solution x = (x 1 , ..., xd )⊤that minimizes a given objective function f : Rd → R, x 7→ f (x).
Any explicit knowledge of f is not given in black-box optimization.
Differential Evolution (DE) is a variant of evolutionary algorithms (EAs) mainly for black-box numerical optimization [40].
The results in the annual IEEE CEC competitions have shown that
DE is competitive with more complex optimizers despite its relative
GECCO ’20, July 8–12, 2020, Cancún, Mexico© 2020 Association for Computing Machinery.
This is the author’s version of the work. It is posted here for your personal use. Not
for redistribution. The definitive Version of Record was published in Genetic andEvolutionary Computation Conference (GECCO ’20), July 8–12, 2020, Cancún, Mexico,
https://doi.org/10.1145/3377930.3389820.

simplicity. A number of previous studies have also demonstrated
the effectiveness of DE in real-world applications [8, 9].
Main control parameters in the basic DE [40] include the population size n, the scaling factor F , and the crossover rate C. Fromthe late 1990s to the early 2000s, it had been believed that the performance of DE is robust with respect to the settings of F and C[40]. However, some studies in the mid-2000s demonstrated thatthe performance of DE is sensitive to the settings of the control
parameters [7, 14, 55]. In general, the performance of EAs significantly depends on the characteristics of a given problem and thestate of the search progress [12]. Thus, a fixed parameter setting(e.g., F = 0.5 and C = 0.9) does not yield the best performanceof an EA. For these reasons, DE algorithms that automatically adjust the control parameters (mainly F and C) have received muchattention in the DE community since the mid-2000s. Representative adaptive DE algorithms include jDE [7], SaDE [37], JADE [53],EPSDE [28], and SHADE [42]. These adaptive DE algorithms havemechanisms to adaptively adjust the F and C parameters during thesearch process. Parameter control methods in EAs can be classified
into deterministic, adaptive, and self-adaptive control methods [12].
Although some DE algorithms with deterministic and self-adaptive
approaches have been proposed (e.g., [32, 48]), adaptive approaches
have mainly been studied in the DE community [45].
As in [44, 45], this paper explicitly distinguishes “an adaptive
DE” and “a parameter adaptation method (PAM) in an adaptive DE”.
While “an adaptive DE” is a complex algorithm that consists ofmultiple components, “a PAM” is a single component only for adaptively adjusting F and C values. As explained in [45], “L-SHADE”[46] is “an adaptive DE” that mainly consists of the following four
components: (a) the current-to-pbest/1 mutation strategy [53], (b)the binomial crossover, (c) the “PAM” in SHADE [42], and (d) the
linear population size reduction strategy. In this paper, we are interested in (c) the “PAM” in SHADE, rather than L-SHADE.
While most previous studies focused on “adaptive DE algorithms”(e.g., [38, 52]), only a few previous studies tried to examine “PAMs”
in DE. Zielinski et al. investigated the performance of some PAMs
for constrained optimization in an isolated manner [54]. Similar
benchmarking studies for multi- and single-objective optimization
were performed in [10, 45], respectively. In [43], a lower bound on
the performance of PAMs was analyzed by using an oracle-based
method for approximating an optimal parameter adaptation processin DE. A simulation framework for quantitatively evaluating theadaptation ability of PAMs was also proposed in [44].
One of the difficulties in analyzing PAMs comes from the unclarity of the control parameter space that consists of F and C. For each
iteration t, a PAM generates a parameter pair of Fit and Cit valuesθ it = (Fit , Cit ) for the i-th individual x it in the population, whereGECCO ’20, July 8–12, 2020, Cancún, Mexico

Ryoji Tanabe

Table 1: Summary of the three landscape analysis.
Landscape analysis

Target space

HeightFitness landscape analysis
Parameter landscape analysis
Adaptive parameter landscape analysis

Solutions of a given problem
Static parameters in an EA (e.g., DE)
Dynamic parameters adjusted by a PAM

Fitness (or objective) values of solutions
Expected performance of an EA with parameters
1-step-lookahead greedy improvement metric (G1) of parametersi ∈ {1, ..., n}. It is desirable that a trial vector u it (a child or a new
solution) generated with θ it is better than its parent x it in termsof their objective values. Generating a good θ it can be viewed asa two-dimensional numerical optimization problem. The goal ofthis problem is to find the optimal parameter pair θ i∗,t ∈ Θit thatminimizes the objective value of a trial vector u it , where Θit ⊆ R2
is a set of all feasible pairs of F and C values. Although the properties of Θit (NOT the optimal parameter) can provide insightful
information about PAMs, they have never been analyzed in the DEcommunity and the evolutionary computation community.
This paper tries to understand Θit by analyzing its adaptive parameter landscape. The term “adaptive parameter landscapes” is a
new concept proposed in this paper inspired by recent work on
parameter landscapes [20, 36, 51]. As reviewed in [27, 31, 35], fitness
landscapes have been well studied in the evolutionary computation
community. In contrast, the field of parameter landscape analysis
is relatively new. A parameter landscape consists of feasible parameter values in an EA. The “height” in parameter landscapes isthe expected performance (or the utility) of an EA with control
parameters on training problem instances [13]. An adaptive parameter landscape proposed in this paper can be viewed as a dynamic
version of a parameter landscape influenced by a PAM. For each
iteration t, each individual x it in the population has its adaptive
parameter landscape that consists of Θit . A PAM can be intuitively
analyzed by investigating its adaptive parameter landscapes. Table 1
summarizes differences in fitness landscapes, parameter landscapes,
and adaptive parameter landscapes. They are explained in Sections
2.3, 2.4, and 3.1, respectively.
Our contributions in this paper are at least threefold:(1) We propose a concept of adaptive parameter landscapes,
which are landscapes of dynamic parameters adjusted by
PAMs. This is the first study to address such dynamicallychanging parameter landscapes in the DE community andthe evolutionary computation community.
(2) We propose a method of analyzing adaptive parameter landscapes using a 1-step-lookahead greedy improvement metric.
(3) We examine adaptive parameter landscapes in three representative PAMs on the 24 BBOB functions [17] by using the
proposed method. Results provide insightful information
about PAMs. Our observations are summarized in Section 6.
The rest of this paper is organized as follows. Section 2 provides
some preliminaries. Section 3 explains the concept of adaptive
parameter landscapes and the proposed analysis method. Section 4
describes the setting of our computational experiments. Section 5shows analysis results. Section 6 concludes this paper.

2

PRELIMINARIESFirst, Section 2.1 explains the basic DE with a PAM. Then, Section
2.2 describes three PAMs in DE (the PAMs in jDE [7], JADE [53], andSHADE [42]). Finally, Sections 2.3 and 2.4 explain fitness landscapeanalysis and parameter landscape analysis, respectively.

2.1

The basic DE with a PAM

Algorithm 1 shows the overall procedure of the basic DE algorithmwith a PAM. Below, we explain DE in an unusual manner for a better
understanding of the proposed G1 metric in Section 3.2.
At the beginning of the search t = 1, the population P t =t{x 1 , ..., x nt } is initialized (line 1), where n is the population size.
For each i ∈ {1, ..., n}, x it is the i-th individual in the population P t .
Each individual represents a d-dimensional solution of a problem.
For each j ∈ {1, ..., d}, x i,t j is the j-th element of x it .
After the initialization of P t , the following steps (lines 2–14) arerepeatedly performed until a termination condition is satisfied. For
each x it , a parameter pair θ it = (Fit , Cit ) is generated by a PAM (line4). The scale factor Fit > 0 controls the magnitude of the mutation.
The crossover rate Cit ∈ [0, 1] controls the number of elements
inherited from x it to a trial vector (child) u it . When θ it is fixed for
all individuals in the entire search process, Algorithm 1 becomesthe classical DE without any PAM [40].
A set of parent indices R = {r 1 , r 2 , ...} are randomly selectedfrom {1, ..., n} \ {i} such that they differ from each other (line 5). For
each x it , a mutant vector v it is generated by applying a differential
mutation to x rt 1 , x rt 2 , ... (line 6). Although a number of mutation
strategies have been proposed in the literature [9], we consider thefollowing two representative mutation strategies:
v it = x rt 1 + Fit (x rt 2 − x rt 3 ),(1)tv it = x it + Fit (x pbest− x it ) + Fit (x rt 1 − x̃ rt 2 ),(2)

where the strategy in (1) is rand/1 [40], and the strategy in (2)is current-to-pbest/1 [53]. The rand/1 strategy is the most basic
strategy, and the current-to-pbest/1 strategy is one of the mostefficient strategies used in recent work (e.g., [42], [46], [53]). Forteach individual, the individual x pbest
is randomly selected from the
top “max(⌊n × p⌋, 2)” individuals in P t , where p ∈ [0, 1] controls
the greediness of current-to-pbest/1. The individual x̃ rt 2 in (2) israndomly selected from a union of P t and an external archive
At , where inferior parent individuals are preserved in At (how to
update At is explained later).
After the mutant vector v it has been generated for each x it , a trial
vector u it is generated by applying crossover to x it and v it (lines
7–9). In this paper, we use binomial crossover [40], which is themost representative crossover method in DE. First, a d-dimensional

Analyzing Adaptive Parameter Landscapes in PAMs for DE
Algorithm 1: The basic DE algorithm with a PAM
t ← 1, initialize P t = {x t1 , ..., x nt } randomly;
2 while The termination criteria are not met do
3
for i ∈ {1, ..., n } do
4
Sample a parameter pair θ it = (F it , C it );5
R ← A set of randomly selected indices from {1, ..., n } \ {i };6v it ← mutation(P t , R, F it );s ← A randomly generated d -dimensional vector (s 1, ..., sd )⊤ ;78
j rand ← A randomly selected number from {1, ..., d };9u it ← crossover(x it , v it , C it , s, j rand );
1

10
11
12
13
14

for i ∈ {1, ..., n } do
if f (u it ) ≤ f (x it ) then x it +1 ← u it ;else x it +1 ← x it ;Update internal parameters for adaptation of F and C ;t ← t + 1;

vector s = (s 1 , ..., sd )T is generated (line 7), where each element ins is randomly selected from [0, 1]. An index j rand is also randomly
selected from {1, ..., d} (line 8). Then, for each i ∈ {1, ..., n}, thetrial vector u it is generated as follows (line 9):(vt
if s j ≤ Cit or j = j randtui, j = i,t j,(3)x i, j otherwise
where the existence of j rand ensures that at least one element isinherited from v it even when Cit = 0.
After the trial vector u it has been generated for each x it , theenvironmental selection is performed in a pair-wise manner (lines
10–12). For each i ∈ {1, ..., n}, x it is compared with u it . The betterone between x it and u it survives to the next iteration t + 1. The
individuals that were worse than the trial vectors are preservedin the external archive A used in (2). When the size of the archive
exceeds a pre-defined size, randomly selected individuals are deleted
to keep the archive size constant. After the environmental selection,some internal parameters in a PAM are updated (line 13).

2.2

Three PAMs for DE

We briefly explain the following three representative PAMs for DE:
the PAM for jDE (P-jDE), the PAM for JADE (P-JADE), and thePAM for SHADE (P-SHADE). Our explanations are based on [45].
Although we briefly explain the three PAMs due to space constraint,
their detailed explanations with precisely described pseudo-codes
can be found in [45]. Below, the generation of the trial vector u it
is said to be successful if f (u it ) ≤ f (x it ) (line 11 in Algorithm 1).
Otherwise, the generation of u it is said to be failed.
P-jDE [7] assigns a pair of Fit and Cit to each x it in P t . At the
beginning of the search, these parameter values are initialized to
Fit = 0.5 and Cit = 0.9 for each i ∈ {1, ..., n}. In each iterationt, Fitrial,t and Citrial,t used for the generation of u it are inheritedfrom x it as follows: Fitrial,t = Fit and Citrial,t = Cit . However, withpre-defined probabilities τ F and τC , these values are randomly generated as follows: Fitrial,t = randu[0.1, 1] and Citrial,t = randu[0, 1].
Here, randu[a, b] is a value selected uniformly randomly from[a, b]. In general, the two hyper-parameters τ F and τC are set to0.1. When the generation of u i,t is successful, Fit +1 = Fitrial,t andCit +1 = Citrial,t . Otherwise, Fit +1 = Fit and Cit +1 = Cit .

GECCO ’20, July 8–12, 2020, Cancún, Mexico
P-JADE [53] adaptively adjusts F and C values using two metaparameters µ F and µC , respectively. For t = 1, both µ F and µC
are initialized to 0.5. For each iteration t, Fit and Cit are generated
as follows: Fit = randc(µ F , 0.1) and Cit = randn(µC , 0.1). Here,randn(µ, σ 2 ) is a value selected randomly from a Normal distribution with mean µ and variance σ 2 . Also, randc(µ, σ ) is a value
selected randomly from a Cauchy distribution with location parameter µ and scale parameter σ . At the end of each iteration,µ F and µC are updated based on sets S F and S C of successful
F and C values as follows: µ F = (1 − c) µ F + c meanL (S F ) andµC = (1 − c) µC + c meanA (S C ). Here, c ∈ [0, 1] is a learning rate.
In general, c = 0.1. While meanA (S F ) is the arithmetic mean of S F ,meanL (S C ) is the Lehmer mean of S C .
P-SHADE [42] adaptively adjusts F and C using historical memF ) and M C = (M C , ..., M C ). Here, H is aories M F = (M 1F , ..., M H
1
Hmemory size. H = 10 was recommended in [44]. For t = 1, all elements in M F and M C are initialized to 0.5. As reviewed in [45], someslightly different versions of P-SHADE have been proposed by thesame authors. As in [45], this paper considers the simplest versionof P-SHADE presented in [44]. In each iteration t, Fit and Cit are generated as follows: Fit = randc(MrF , 0.1) and Cit = randn(MrC , 0.1),
where r is an index randomly selected from {1, ..., H }. At the endof each iteration, the k-th elements in M F and M C are updated
as follows: MkF = meanL (S F ) and MkC = meanL (S C ). An indexk ∈ {1, ..., H } represents the position to be updated and is incremented on every update. If k > H , k is re-initialized to 1.

2.3

Fitness landscapeAccording to Pitzer and Affenzeller [35], a fitness landscape Lf in anumerical optimization problem is defined by a 3-tuple as follows:
Lf = (X, f , D),(4)where X ⊆ Rd is the solution space (i.e., a set of all feasible solutionsx). Also, f : x 7→ f (x) is the objective function of a given problem.
D : x × x 7→ R is a distance function between two solutions (e.g.,
the Euclidean distance).
An analysis of Lf can provide useful information even for blackbox optimization. For example, if the features of a given problem(e.g., ruggedness and neutrality) becomes clear by analyzing Lf , anappropriate optimizer can be selected [31]. A number of methods
for analyzing Lf have been proposed in the literature [27, 35]. Representative methods include fitness distance correlation (FDC) [24],dispersion metric (DISP) [26], and evolvability [39]. Recently, moresophisticated methods have been proposed, such as exploratory
landscape analysis (ELA) [29] and local optima networks (LON) [1].
These methods can quantify at least one feature about Lf . For example, the FDC value represents a global structure of Lf based on
the correlation between the distance from solutions to the optimal
solution and their objective values.

2.4

Parameter landscape

Roughly speaking, a parameter tuning problem [13] involves finding a tuple θ of control parameters that optimizes the empiricallyestimated performance of an algorithm on a set of training problem
instances. For example, a parameter tuning problem for the basic
DE with no PAM can be defined as a problem to find θ = (F , C)GECCO ’20, July 8–12, 2020, Cancún, Mexico

Ryoji Tanabethat minimizes the average objective values of best-so-far solutions
on the Sphere, Rastrigin, and Rosenbrock functions. In general, theparameter tuning problem addresses only numerical parameters.
In contrast, an algorithm configuration problem [22] addresses
numerical, ordinal (e.g., low, medium, and high), and categoricalparameters (e.g., the choice of mutation strategies). According to[21], parameter tuning is a problem that involves only numerical parameters, while algorithm configuration is a problem that involves
many categorical parameters. The parameter tuning problem canbe viewed as a special case of the algorithm configuration problem.
Parameter landscapes appear in parameter tuning problems.
Since it is difficult to define a distance function for categorical parameters, the field of parameter landscape analysis considers only
numerical parameters as in [36]. The term “parameter landscapes”
was first coined in [51]. Parameter landscapes were also denoted
as “performance landscapes” [50], “meta-fitness landscapes” [34],“utility landscapes” [13], “ERT landscapes” [5], “parameter configuration landscapes” [20], and “algorithm configuration landscapes”[36]. To avoid any confusion, we use the term “parameter landscapes” throughout this paper. Since only numerical parameters
are considered, we believe that the term “parameter landscapes” isappropriate. Although some previous studies (e.g., [4, 25]) did not
use the term “landscapes”, they essentially investigated parameter
landscapes.
According to Harrison et al. [20], a parameter landscape Lp in aparameter tuning problem is formally defined as follows:
Lp = (Θ, M, D),(5)where the definition of Lp in (5) is a slightly different versionof the original one in [20]. Θ is the numerical parameter space(i.e., a set of all feasible parameters θ ). Also, M : θ 7→ M(θ ) is a
performance metric that empirically estimates the performanceof a given algorithm on a set of training problem instances (e.g.,
the average of objective values [20] and PAR10 [36]). Similar to (4),D : θ × θ 7→ R is a distance function between two parameters.
Helpful information about parameter tuning and an algorithm
can be obtained by analyzing Lp . For example, as mentioned in [50],if Lp is multimodal, a global parameter tuner may perform better
than a local parameter tuner. As demonstrated in [51], an influenceof multiple parameters on the performance of an algorithm can bevisually discussed by analyzing Lp .

3

PROPOSED ADAPTIVE PARAMETERLANDSCAPE ANALYSISFirst, Section 3.1 explains the proposed concept of adaptive parameter landscapes. Then, Section 3.2 introduces a 1-step greedy
fitness improvement (G1) metric, which is a performance metricfor adaptive parameter landscapes. Finally, Section 3.3 proposes themethod of analyzing adaptive parameter landscapes.

3.1

Adaptive parameter landscape

We define an adaptive parameter landscape La in a PAM as follows:
La = (Θit , M, D),

The difference between Lp in (5) and La in (6) is only the target
space (Θ vs. Θit ). While Θ in Lp is static, Θit in La is dynamic. An
adaptive parameter landscape can be viewed as a parameter landscape that captures a moment in a parameter adaptation process.
Our ultimate goal is to understand PAMs for DE. While there
have been significant contributions in analyzing DE itself in recent
years (e.g., [3, 33]), only a few previous studies examined PAMs for
DE (see Section 1). One reason is that very little is known about
dynamically changing parameter spaces handled by PAMs. We
believe that this issue can be addressed by analyzing adaptive parameter landscapes. A better understanding of adaptive parameter
landscapes in PAMs can also lead to design a more efficient PAM.
Recall that Table 1 in Section 1 has already summarized the differences in a fitness landscape Lf in (4), a parameter landscape Lp
in (5), and an adaptive parameter landscape La in (6). Very recently,
Jankovic and Doerr [23] investigated dynamic fitness landscapes
seen from CMA-ES [18]. They denoted their analysis as “adaptive
landscape analysis”. While “adaptive landscape analysis” focuses
on fitness landscapes of a problem, our adaptive parameter landscape analysis focuses on dynamic parameter landscapes adaptively
adjusted by PAMs. Thus, the names “adaptive landscape analysis”
and “adaptive parameter landscape analysis” are similar, but they
are totally different from each other. As analyzed in [6, 11, 19], thebest parameter settings in EAs depend on the maximum numberof function evaluations when the performance of EAs is estimatedbased on final results (e.g., the objective value of the best-so-far
solution at the end of each run). We are interested in dynamicallychanging parameter landscapes, rather than such static parameter
landscapes limited by a termination criterion.
Automated algorithm methods based on fitness landscape features of a given problem have been well studied in the evolutionary
computation community [31]. Note that an adaptive parameter
landscape analysis do not mean such a parameter selection approach that seeks the best static parameters (i.e., F and C, not Fit
and Cit ) based on fitness landscape features in a one-shot manner.
For example, this paper is unrelated to [5].

3.2

1-step-lookahead greedy improvement
metric (G1)

One critical obstacle in analyzing an adaptive parameter landscape
La in (6) is how to define the performance metric M. For an analysis
of a parameter landscape Lp in (5), some performance metrics
can be derived from the field of parameter tuning without any
significant change (e.g., PAR10 as mentioned in Section 2.4). In
contrast, M in La is not obvious.
Here, we introduce a 1-step-lookahead greedy improvement (G1)
metric as a performance metric M for analyzing La . As explainedin Section 2.1 using Algorithm 1, DE generates the trial vector u it
for each parent individual x it (i ∈ {1, ..., n}) at each iteration t.
A parameter pair of F and C values θ it = (Fit , Cit ) is used for thegeneration of u it . We define the G1 value of θ it as follows:(6)

where Θit is the numerical parameter space for the i-th individual in
the population at iteration t (i.e., a set of all feasible parameters θ it ).

G1(θ it ) =(f (x it ) − f (u it )0

if f (u it ) < f (x it ).
otherwise(7)

Analyzing Adaptive Parameter Landscapes in PAMs for DE

3.3

Method for analyzing La

We explain the proposed method of analyzing La . Our proposed
method can be incorporated into DE in Algorithm 1 with no change.
The procedure of our proposed method is totally independent fromthat of DE. Thus, the search behavior of DE with and without our
proposed method is exactly the same.
t , ..., θ tFirst, m parameter pairs θ i,1
i,m are generated for eachtindividual x i at iteration t (line 4 in Algorithm 1). Although any
generation method can be used (e.g., the random sampling method),
we generate m parameter pairs in a grid manner in this study.
We generate 50 × 50 parameter pairs in the ranges F ∈ [0, 1] andC ∈ [0, 1]. Thus, m = 50 × 50 = 2 500 in this study. Figure 1(a)shows the distribution of the 50×50 parameter pairs. We notice thatany differential mutation strategy with F = 0 does not work well,resulting poor performance of DE. Just for the sake of simplicity,
we include F = 0 in the set of parameter pairs. Since the G1 value ofpoor parameter pairs is 0, the inclusion F = 0 does not significantly
influence our analysis of La . We also notice that the upper valueof F is unbounded in principle, but it was generally set to 1 in mostprevious studies (e.g., [7, 28, 42, 53]).
Then, we calculate the G1 values of the m parameter pairst ), ..., G1(θ t ) by simply generating m trial vectors u t ,G1(θ i,1i,mi,1t . Their objective values f (u t ), ..., f (u t ) are evaluated..., u i,mi,mi,1t ) is calculated by (7). It
by f . Then, for each j ∈ {1, ..., m}, G1(θ i,jshould be noted that m extra function evaluations by f are neededt ), ..., f (u t ).
to calculate the m objective values f (u i,1i,m
In the proposed method, the m extra function evaluations foreach individual are not counted in the function evaluations used in
the search. This manner is similar to GAO [43]. The m trial vectors
are used only for adaptive parameter landscape analysis and are
not used for the actual search. Independently of the generation of
the m trial vectors, each individual x it generates its trial x it as in
the traditional DE (lines 4–9 in Algorithm 1). As mentioned above,
the behavior of DE with any PAM does not change even whengenerating the m extra trial vectors.

1

1

0.8

0.8

0.6

0.6

0.60

0.4

0.45

1.00
0.90
0.75

C

C

The G1(θ it ) value in (7) represents how significantly θ it contributes to generate a better u it than x it in terms of the objective
value. A large G1 value indicates that the corresponding θ it cangenerate a good u it . For example, let us consider the following threet , θ t , and θ t .
parameter pairs used for the generation of u it : θ i,1
i,2
i,3
t
t ) = 1.2,
Their G1 values are also as follows: G1(θ i,1 ) = 0.7, G1(θ i,2t ) = 0. In this case, θ t is the best in the three parameter
and G1(θ i,3i,2
pairs in terms of G1. The objective value of x it can be significantly
t . G1(θ t ) = 0 means that u t generated by
improved by using θ i,2i,3itusing θ i,3 is inferior (or equal) to x it . Note that the G1 value isalways non-negative. In DE, an inferior trial vector compared toits parent individual cannot survive to the next iteration. For thisreason, we equally treat all parameter pairs of F and C values thatgenerate worse trial vectors than their parent individuals.
The idea of measuring the fitness improvement value as in (7)itself is not new at all. Such an approach can be found in the literature (e.g., [16, 49]). In contrast to previous studies, the G1 metricaims to capture adaptive parameter landscapes by the proposed
method explained in the next section.

GECCO ’20, July 8–12, 2020, Cancún, Mexico

0.4
0.2

0.30

0.2

0
0 0.2 0.4 0.6 0.8 1
F(a) 50 × 50 pairs

0.15

0
0 0.2 0.4 0.6 0.8 1
F

0.00(b) Contour map of La

Figure 1: (a) Distribution of 50 × 50 pairs of F and C values
generated in a grid manner. (b) Contour map of its La .
The stochastic nature of the basic DE is due to (1) the random
selection of individual indices R = {r 1 , r 2 , ...} for mutation (line
5 in Algorithm 1) and (2) the generation of random numbers s =(s 1 , ..., sd )⊤ and j rand for crossover (lines 7–8 in Algorithm 1). Thus,
the stochastic nature of DE can be “virtually” suppressed by fixing
these random factors (R, s, and j rand ). For each iteration t, eachindividual x it generates one actual trial vector u it and the m extra
t , ..., u t
trial vectors u i,1i,m using the same R, s, and j rand . When usingt
the current-to-pbest/1 strategy in (2), x pbest
must also be fixed. Notethat this suppression mechanism is used only to generate the mtrial vectors for each individual x it .
Figure 1(b) shows the contour map of La based on the 50 × 50
parameter pairs in Figure 1(a). The height in La is the normalizedG1 value. For the sake of clarity, for each individual x it , we normalize
all G1 values into the range [0, 1] by using the maximum G1 valuet ) =G1max and the minimum G1 value G1min as follows: G1(θ i,jt ) − G1min )/(G1max − G1min ), where j ∈ {1, ..., m}. The(G1(θ i,j
contour map in Figure 1(b) is La in the 100-th individual in PSHADE on the 20-dimensional f 1 in the BBOB function set [17].
Figure 1(b) shows La at the 100-th function evaluations. We used
the same experimental setting explained in Section 4. Details of thesetting are described in Section 4 later. Figure 1(b) is the same withthe most bottom left of Figure 2. In Figure 1(b), the parameter pairof F = 0.78 and C = 1 is the best in the m = 50 × 50 parameter pairs
in terms of G1. As seen from Figure 1(b), the closer the parameter
pair is to the best parameter pair, the better the G1 value.

4

EXPERIMENTAL SETUP

We performed all experiments using the COCO software (https://github.com/numbbo/coco), which is standard in the GECCO blackbox optimization benchmarking (BBOB) workshops since 2009. We
used the 24 BBOB noiseless functions f 1 , ..., f 24 [17], which aregrouped into the following five categories: separable functions(f 1 , ..., f 5 ), functions with low or moderate conditioning (f 6 , ..., f 9 ),functions with high conditioning and unimodal (f 10 , ..., f 14 ), multimodal functions with adequate global structure (f 15 , ..., f 19 ), andmultimodal functions with weak global structure (f 20 , ..., f 24 ). The
dimensionality d of the BBOB functions was set to 2, 3, 5, 10, 20,
and 40. For each problem instance, 15 runs were performed. Thesesettings strictly adhere to the standard benchmarking procedure in

GECCO ’20, July 8–12, 2020, Cancún, Mexico

100fe

8,000fe1

12,000fe

1

16,000fe

0

0

0
1 0
1

0
1 0
1

0
1 0
1

1

43,000fe 65,000fe

87,000fe

1

1

0
1 0
1

0
1 0
1

0
1 0
1

1

0
1 0
1

0
1 0
1

0
1 0
1

1

0
1 0

0
1 0

0
1 0

1

0

0
1 0
1

0
1 0
1

1

0

0

1

1

1

0
1 0
1

0

0
1 0
1

0
1 0
1

0
1 0
1

1

0

0
1 0

0
1 0

0
1 0

1

0

0
1 0

0

1

50th

0

0

0

75th

1

75th

1

0

0

0

100th

1

100th

1

0

0
0

0.15 0.3 0.45 0.6 0.75 0.9 1

G1

x axis: Fy axis: C

Figure 2: Contour maps of adaptive parameter landscapes in
P-SHADE on f 1 with d = 20.
the GECCO BBOB workshops. The maximum number of function
evaluations was set to 10 000 × d.
We analyze the three PAMs (P-jDE, P-JADE, and P-SHADE)
explained in Section 2.2. Source code used in this study can be
downloaded from https://github.com/ryojitanabe/APL. We set their
hyper-parameters to the values recommended by the corresponding
articles. As in [7, 42, 53], we set the population size n to 100. We used
the rand/1 and current-to-pbest/1 strategies described in Section
2.1. However, we show only results with current-to-pbest/1 due tospace constraints. As in [53], the control parameters of the currentto-pbest/1 strategy were set as follows: p = 0.05 and |A| = n. We
used binomial crossover.

RESULTSThis section analyzes adaptive parameter landscapes in PAMs for
DE by the proposed method. Our findings are summarized in Section 6. Section 5.1 discusses the shapes of adaptive parameter landscapes by using contour maps as in Figure 1(b). Section 5.2 examines
adaptive parameter landscapes using landscape measures.

5.1

1

1

50th

1

5

100fe

1

25th

1

25th

1

Ryoji Tanabe

Analysis with contour maps

Let us consider that a DE with the population size n = 100 terminates the search at 10 000 function evaluations on a problem. In
this case, we can obtain 9 900 contour maps, where the first 100000.15 0.3 0.45 0.6 0.75 0.9 1

G1

x axis: Fy axis: C

Figure 3: Contour maps of adaptive parameter landscapes in
P-SHADE on f 3 with d = 20.

evaluations out of 10 000 are for the initialization of the population. Also, we performed 15 runs of the 3 PAMs on the 24 BBOB
functions with the 6 dimensionalities d ∈ {2, 3, 5, 10, 20, 40}. Even
if all runs terminate at 10 000 function evaluations, we can obtain
64 152 000 contour maps (= 9 900 × 15 × 3 × 24 × 6) contour maps.
It is impossible and meaningless to show 64 152 000 contour maps
in this paper.
For the above-mentioned reason, we “thinned” data as follows
so that we can focus only on meaningful results.
• Data of all runs. For each PAM, we show results of a single runwith a median best-so-far error value, which is the gap between theobjective values of the best-so-far solution and the optimal solution.
When the error value is smaller than 10−8 , it is treated as 0. Ties
are broken by the number of function evaluations that is used tofind the best-so-far solution.
• Data of all individuals. For each iteration, first, all individuals are
sorted based on their objective values in descending order. Then,
we show only adaptive parameter landscapes of the 25th, 50th, 75th,
and 100th individuals out of 100 individuals. Since a parameter pair
for the best (1st) individual is seldom successful, we omit its results.
The reason is discussed in Section 5.2 later.
• Data of all function evaluations. In order to reduce the computational cost of the proposed method, we calculate adaptive parameter
landscapes only in every 1 000 function evaluations. Also, we show

Analyzing Adaptive Parameter Landscapes in PAMs for DE

1

1

1.00
0.90

0.8
0.4

0.45
0.30

0.2

0.15

0
0 0.2 0.4 0.6 0.8 1
F(a) f 22

0.00

0.75

0.6

0.60

0.4

0.45

C

0.60

1.00
0.90

0.8

0.75

0.6

C

results at 100, ⌊0.5 festop ⌋, ⌊0.75 festop ⌋, and ⌊1 festop ⌋ function evaluations, where festop is the number of function evaluations when
the best-so-far solution is updated last time.
Figures 2 and 3 show the contour maps of adaptive parameter
landscapes in P-SHADE on f 1 and f 3 with d = 20, respectively.
Here, f 1 and f 3 are modified versions of the Sphere function andthe Rastrigin function, respectively. In Figures 2 and 3, “fe” stands
for “function evaluations”. The x and y axes represent F and C,
respectively. The star in each figure is the best parameter pair thatmaximizes the G1 value. The circle in each figure is the parameter
pair generated by the PAM. See Section 3.3 for how to generate
Figures 2 and 3. When the G1 values of all 50×50 parameter pairs are
0 (i.e., no parameter pair can improve the individual), the adaptive
parameter landscape is flat. In such a case, we do not show results(e.g., the result of the 50th individual at 43 000 function evaluationsin Figure 3). Figures S.73 in the supplementary file show error
values of P-jDE, P-JADE, and P-SHADE on all 24 BBOB functions
with d = 20. Note that we are not interested in benchmarking DE
algorithms. As shown in Figures S.73(a) and (c), P-SHADE found the
optimal solution on f 1 and f 3 at about 16 000 and 87 000 function
evaluations in a median run. Due to space constraints, we show
results of P-jDE, P-JADE, and P-SHADE on the 24 BBOB functions(f 1 , ..., f 24 ) with d = 20 in Figures S.1–S.72 in the supplementary
file. Although we show only the results of P-SHADE in this section,
the qualitative results of the three PAMs are similar. The results on
the functions with d ≥ 5 are also similar to those with d = 20.
Below, we discuss the shape of adaptive parameter landscapes
obtained in this study. Readers who want to quickly know ourobservations can refer to Section 6. As shown in Figures 2 and 3,
the shape of adaptive parameter landscapes is different dependingon the search progress. Since improving randomly initialized individuals is easy, the area with non-zero G1 values is large at the
beginning of the search in most cases. This means that generating
a successful parameter pair of F and C is easy for PAMs in an earlystage of evolution. However, the area with non-zero G1 values decreases as the search progresses. Thus, it is relatively difficult togenerate a parameter pair of F and C that improves each individual
in a mid stage of evolution. As seen from Figures 2 and 3, the shapeof adaptive parameter landscapes is also different depending on
the rank of each individual. By comparing Figures 2 and 3, we cansee that generating a successful parameter pair on a multimodal
function is more difficult than that on a unimodal function. Adaptive parameter landscapes at 43 000 and 65 000 function evaluationsin Figure 3 indicate that the area with non-zero G1 values is verysmall like needle-in-haystack landscapes. In addition to the multimodality, the nonseparability is an important factor to determine
the shape of adaptive parameter landscapes as seen from results on
nonseparable unimodal functions (f 6 –f 14 ) shown in Figures S.6–
S.62 in the supplementary file. Interestingly, as shown in adaptive
parameter landscapes at 87 000 function evaluations in Figure 3, thearea with non-zero G1 values becomes large again in a late stageof evolution. This is because the population has well converged tothe optimal solution, and generating better trial vectors is not so
difficult at such a situation on f 3 .
The shape of adaptive parameter landscapes is significantly influenced by the global structures of fitness landscapes. For example,

GECCO ’20, July 8–12, 2020, Cancún, Mexico

0.30

0.2

0.15

0
0 0.2 0.4 0.6 0.8 1
F

0.00(b) f 23

Figure 4: Contour maps of adaptive parameter landscapes in
P-SHADE on f 22 and f 23 with d = 20.
Figure 4 shows that the contour maps of adaptive parameter landscapes in the 100-th individual of P-SHADE on f 22 and f 23 withd = 20 at 100 function evaluations. Figures 4(a) and (b) are partsof Figures S.70 and S.71 in the supplementary file, respectively.
The original functions of f 22 and f 23 are the Gallagher’s Gaussian
21 peaks function and the Katsuura function, which have fitness
landscapes without any global structure. Figure 4(a) shows that theadaptive parameter landscape on f 22 has only the small area with
non-zero G1 values even at the beginning of the search. Figure 4(b)also shows that the adaptive parameter landscape do not have any
global structure similar to the fitness landscape of f 23 .
As seen from the positions of the star and the circle in Figures
2 and 3, a parameter pair actually generated by P-SHADE is far
from the best parameter pair. Ideally, it is desirable that a PAM cangenerate a parameter pair close to the best parameter pair. Thisobservation indicates that there is room for improving PAMs in DE.

5.2

Analysis using landscape measures

This section analyzes adaptive parameter landscapes using two
representative landscape measures (FDC [24] and DISP [26]) anda non-zero ratio (NZR) measure. FDC measures the correlation
between objective values and the distance to the best solution
found (or the optimal solution). A large FDC value indicates thatthe corresponding fitness landscape has a strong global structure.
In DISP, first, all solutions are sorted based on their objective values
in descending order. Then, the dispersion of the top b solutions is
calculated based on the average pairwise distance between them(b = ⌊0.1 m⌋ in this study). A large DISP value indicates that thecorresponding fitness landscape has a multi-funnel. Although FDC
and DISP were originally proposed for fitness landscape analysis,
they can be extended for parameter landscape analysis with nosignificant change, as demonstrated in [20]. When using FDC andDISP for adaptive parameter landscape analysis, “the objective value”
is replaced with the G1 value, and “the solution” is replaced with theparameter pair of F and C values. We did not normalize parametervalues since F ∈ [0, 1] and C ∈ [0, 1]. FDC and DISP perform
poorly in high-dimensional spaces [30], but we address only thetwo-dimensional space (F and C).
We introduce NZR for analyzing adaptive parameter landscapes.
We do not argue that NZR is one of our contributions since it justcounts numbers. The NZR value of an adaptive parameter landscapeGECCO ’20, July 8–12, 2020, Cancún, Mexico

75th

100th

1st

DISP

FDC

0.6
0.4

0.0

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

0.2

24 BBOB functions(a) FDC

25th

50th

75th

100th

1st

0.5

0.9

0.4

0.6

0.3
0.2

25th

50th

75th

100th

0.3
0.0

24 BBOB functions

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

50th

NZR

25th

0.8

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

1st

Ryoji Tanabe(b) DISP

24 BBOB functions(c) NZR

Figure 5: Average FDC, DISP, and NZR values of adaptive parameter landscapes in P-SHADE (d = 20).
in the i-th individual at iteration t is given as follows:
o
1 n t
t
t
t
NZR(θ i,1
, ..., θ i,m)=
θ i, j |G1(θ i,) > 0, j = 1, ..., m , (8)jmwhere the NZR value is always in the range [0, 1]. NZR measuresthe difficulty in generating a “successful” parameter pair of F and C
values based on the area with non-zero G1 values (see Section 2.2 forthe definition of “successful”). A large NZR value indicates that it iseasy to generate a successful parameter pair on the corresponding
adaptive parameter landscape.
Figure 5 shows the average FDC, DISP, and NZR values of the
1st, 25th, 50th, 75th, and 100th individuals in P-SHADE at 100, 1 000,
2 000, ... function evaluations on the 24 BBOB functions with d = 20.
Figures S.74–S.82 in the supplementary file show results of thethree PAMs on the functions with d ∈ {2, 3, 5, 10, 20, 40}.
Figure 5(a) shows that all FDC values are non-negative on allfunctions. As seen from results on all 24 functions, the worse theindividual is, the larger the FDC value. Adaptive parameter landscapes for individuals with similar ranks (e.g., the 50th and 75th
individuals) have similar FDC values. This observation indicatesthat the global structures of adaptive parameter landscapes cancorrelate with the rank of individuals. Some previous studies (e.g.,[15, 41, 47]) gave a rule of thumb that the appropriate parameter pairof F and C values may depend on the rank of individuals. Although
this rule of thumb has never been supported by any result, it canbe justified by our observation in adaptive parameter landscapes.
Results of NZR in Figure 5(c) show that generating a successful
parameter pair is relatively easy for inferior individuals. Ali [2]demonstrated that generating a better trial vector than an inferior
individual in the population is easy. We believe that our observation
supports a generalization of Ali’s observation since it can be appliedeven to adaptive parameter landscapes.
As shown in Figure 5(a), the FDC value is different depending on
the function. While the average FDC values on the three separable
and unimodal functions (f 1 , f 2 , and f 5 ) are large, those on the multimodal or nonseparable functions are small, except for f 21 and f 22 .
Although f 21 and f 22 are multi-modal functions, each peak of theirfitness landscapes is unimodal. This property of f 21 and f 22 may
influence the FDC values of adaptive parameter landscapes.
As shown in Figures 5(a) and (b), results of DISP are consistent
with the above-mentioned results of FDC in most cases. Figure 5(b)can be viewed as an upside-down version of Figure 5(a). This may
be because both FDC and DISP quantify the global structures offitness landscapes. An analysis with other landscape measures (e.g.,
ELA [29] and LON [1]) is another future work.
Figures S.74–S.82 in the supplementary file show that the resultsof P-jDE, P-JADE, and P-SHADE for d ∈ {5, 10, 40} are similar to
those for d = 20 (Figure 5). In contrast, results for d ∈ {2, 3} arenoisy. This may be because DE algorithms with PAMs do not work
well on such low-dimensional problems as reported in [45].

6

CONCLUSION

We have analyzed adaptive parameter landscapes based on F and C
in PAMs for DE. We introduced the concept of adaptive parameter
landscapes (Section 3.1). We proposed the method of analyzing
adaptive parameter landscapes based on the G1 metric (Sections
3.2 and 3.3). We also examined adaptive parameter landscapes in
P-jDE, P-JADE, and P-SHADE on the 24 BBOB functions by using
the proposed method (Section 5).
Our observations in this study can be summarized as follows:i) An adaptive parameter landscape La (NOT the optimal parameter θ ∗ ) is different depending on the search progress.
For example, it is relatively easy to generate successful parameters in an early stage of evolution.
ii) La (NOT θ ∗ ) is significantly influenced by the characteristics of a given problem (e.g., the local/global multimodality).
iii) La (NOT θ ∗ ) differs depending on the rank of an individual,
but La of individuals with similar ranks are generally similar.
iv) In most cases, P-jDE, P-JADE, and P-SHADE generate aparameter pair of F and C values far from the best parameter
pair. This means that there is room for improving PAMs.
We emphasize that our observations about PAMs could not be
obtained without analyzing adaptive parameter landscapes. We
believe that our observations can be useful clues to design an efficient PAM. Overall, we conclude that adaptive parameter landscapeanalysis can provide important information about PAMs for DE.
Although we examined adaptive parameter landscapes in PAMs
for DE, we believe that the proposed analysis method can be applied to PAMs for other evolutionary algorithms, including genetic
algorithms and evolution strategies. Further analysis is needed.

ACKNOWLEDGMENTS
This work was supported by Leading Initiative for Excellent Young
Researchers, MEXT, Japan.

Analyzing Adaptive Parameter Landscapes in PAMs for DE

REFERENCES[1] J. Adair, G. Ochoa, and K. M. Malan. 2019. Local optima networks for continuous
fitness landscapes. In GECCO (Companion). 1407–1414.
[2] M. M. Ali. 2011. Differential evolution with generalized differentials. J. Comput.
Appl. Math. 235, 8 (2011), 2205–2216.
[3] J. Arabas and D. Jagodzinski. 2020. Toward a Matrix-Free Covariance Matrix
Adaptation Evolution Strategy. IEEE TEVC 24, 1 (2020), 84–98.
[4] T. Bartz-Beielstein, C. Lasarczyk, and M. Preuss. 2010. The Sequential Parameter
Optimization Toolbox. In Experimental Methods for the Analysis of Optimization
Algorithms. 337–362.
[5] N. Belkhir, J. Dréo, P. Savéant, and M. Schoenauer. 2016. Feature Based Algorithm
Configuration: A Case Study with Differential Evolution. In PPSN. 156–166.
[6] L. C. T. Bezerra, M. López-Ibáñez, and T. Stützle. 2018. A Large-Scale Experimental Evaluation of High-Performing Multi- and Many-Objective Evolutionary
Algorithms. Evol. Comput. 26, 4 (2018).
[7] J. Brest, S. Greiner, B. Bošković, M. Mernik, and V. Žumer. 2006. Self-Adapting
Control Parameters in Differential Evolution: A Comparative Study on Numerical
Benchmark Problems. IEEE TEVC 10, 6 (2006), 646–657.
[8] S. Das, S. S. Mullick, and P. N. Suganthan. 2016. Recent advances in differential
evolution - An updated survey. Swarm and Evol. Comput. 27 (2016), 1–30.
[9] S. Das and P. N. Suganthan. 2011. Differential Evolution: A Survey of the Stateof-the-Art. IEEE TEVC 15, 1 (2011), 4–31.
[10] M. Drozdik, H. E. Aguirre, Y. Akimoto, and K. Tanaka. 2015. Comparison of
Parameter Control Mechanisms in Multi-objective Differential Evolution. In LION.
89–103.
[11] A. S. D. Dymond, A. P. Engelbrecht, and P. S. Heyns. 2011. The sensitivity ofsingle objective optimization algorithm control parameter values under different
computational constraints. In IEEE CEC. 1412–1419.
[12] A. E. Eiben, R. Hinterding, and Z. Michalewicz. 1999. Parameter control in
evolutionary algorithms. IEEE TEVC 3, 2 (1999), 124–141.
[13] A. E. Eiben and S. K. Smit. 2011. Parameter tuning for configuring and analyzing
evolutionary algorithms. Swarm and Evol. Comput. 1, 1 (2011), 19–31.
[14] R. Gämperle, S. D. Müller, and P. Koumoutsakos. 2002. A Parameter Study for
Differential Evolution. In Int. Conf. on Adv. in Intelligent Systems, Fuzzy Systems,
Evol. Comput. 293–298.
[15] A. Ghosh, S. Das, A. Chowdhury, and R. Giri. 2011. An improved differential
evolution algorithm with fitness-based adaptation of the control parameters. Inf.
Sci. 181, 18 (2011), 3749–3765.
[16] N. Hansen, D. V. Arnold, and A. Auger. 2015. Evolution Strategies. Springer.
[17] N. Hansen, S. Finck, R. Ros, and A. Auger. 2009. Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions. Technical Report.
INRIA.
[18] N. Hansen and A. Ostermeier. 2001. Completely Derandomized Self-Adaptation
in Evolution Strategies. Evol. Comput. 9, 2 (2001), 159–195.
[19] K. R. Harrison, A. P. Engelbrecht, and B. M. Ombuki-Berman. 2018. Optimal
parameter regions and the time-dependence of control parameter values forthe particle swarm optimization algorithm. Swarm and Evol. Comput. 41 (2018),
20–35.
[20] K. R. Harrison, B. M. Ombuki-Berman, and A. P. Engelbrecht. 2019. The Parameter
Configuration Landscape: A Case Study on Particle Swarm Optimization. In IEEE
CEC. 808–814.
[21] H. H. Hoos. 2012. Automated Algorithm Configuration and Parameter Tuning.
In Autonomous Search. 37–71.
[22] F. Hutter, H. H. Hoos, K. Leyton-Brown, and T. Stützle. 2009. ParamILS: An
Automatic Algorithm Configuration Framework. JAIR 36 (2009), 267–306.
[23] A. Jankovic and C. Doerr. 2019. Adaptive landscape analysis. In GECCO (Companion). 2032–2035.
[24] T. Jones and S. Forrest. 1995. Fitness Distance Correlation as a Measure of Problem
Difficulty for Genetic Algorithms. In ICGA. 184–192.
[25] I. Loshchilov, M. Schoenauer, and M. Sebag. 2012. Alternative Restart Strategies
for CMA-ES. In PPSN. 296–305.
[26] M. Lunacek and D. Whitley. 2006. The dispersion metric and the CMA evolution
strategy. In GECCO. 477–484.
[27] K. Malan and A. P. Engelbrecht. 2013. A survey of techniques for characterising
fitness landscapes and some possible ways forward. Inf. Sci. 241 (2013), 148–163.
[28] R. Mallipeddi, P. N. Suganthan, Q. K. Pan, and M. F. Tasgetiren. 2011. Differential
evolution algorithm with ensemble of parameters and mutation strategies. Appl.

GECCO ’20, July 8–12, 2020, Cancún, Mexico
Soft Comput. 11 (2011), 1679–1696.
[29] O. Mersmann, B. Bischl, H. Trautmann, M. Preuss, C. Weihs, and G. Rudolph.
2011. Exploratory landscape analysis. In GECCO. 829–836.
[30] R. Morgan and M. Gallagher. 2014. Sampling Techniques and Distance Metrics
in High Dimensional Continuous Landscape Analysis: Limitations and Improvements. IEEE TEVC 18, 3 (2014), 456–461.
[31] M. A. Muñoz, Y. Sun, M. Kirley, and S. K. Halgamuge. 2015. Algorithm selection
for black-box continuous optimization problems: A survey on methods andchallenges. Inf. Sci. 317 (2015), 224–245.
[32] M. G. H. Omran, A. A. Salman, and A. P. Engelbrecht. 2005. Self-adaptive Differential Evolution. In CIS. 192–199.
[33] K. R. Opara and J. Arabas. 2019. Differential Evolution: A survey of theoreticalanalyses. Swarm and Evol. Comput. 44 (2019), 546–558.
[34] M. E. H. Pedersen. 2010. Tuning & Simplifying Heuristical Optimization. Ph.D.
Dissertation. University of Southampton.
[35] E. Pitzer and M. Affenzeller. 2012. A Comprehensive Survey on Fitness LandscapeAnalysis. In Recent Advances in Intelligent Engineering Systems. 161–191.
[36] Y. Pushak and H. H. Hoos. 2018. Algorithm Configuration Landscapes: - More
Benign Than Expected?. In PPSN. 271–283.
[37] A. K. Qin, V. L. Huang, and P. N. Suganthan. 2009. Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization. IEEE TEVC
13, 2 (2009), 398–417.
[38] C. Segura, C. A. C. Coello, E. Segredo, and C. León. 2014. An analysis of theautomatic adaptation of the crossover rate in differential evolution. In IEEE CEC.
459–466.
[39] T. Smith, P. Husbands, P. J. Layzell, and M. O’Shea. 2002. Fitness Landscapes andEvolvability. Evol. Comput. 10, 1 (2002), 1–34.
[40] R. Storn and K. Price. 1997. Differential Evolution - A Simple and Efficient
Heuristic for Global Optimization over Continuous Spaces. J. Glo. Opt. 11, 4(1997), 341–359.
[41] T. Takahama and S. Sakai. 2012. Efficient Constrained Optimization by the ϵConstrained Rank-Based Differential Evolution. In IEEE CEC. 1–8.
[42] R. Tanabe and A. Fukunaga. 2013. Success-History Based Parameter Adaptation
for Differential Evolution. In IEEE CEC. 71–78.
[43] R. Tanabe and A. Fukunaga. 2016. How Far Are We from an Optimal, Adaptive
DE?. In PPSN. 145–155.
[44] R. Tanabe and A. Fukunaga. 2017. TPAM: a simulation-based model for quantitatively analyzing parameter adaptation methods. In GECCO. 729–736.
[45] R. Tanabe and A. Fukunaga. 2020. Reviewing and Benchmarking Parameter
Control Methods in Differential Evolution. IEEE Trans. Cyber. 50, 3 (2020), 1170–
1184.
[46] R. Tanabe and A. S. Fukunaga. 2014. Improving the search performance of SHADE
using linear population size reduction. In IEEE CEC. 1658–1665.
[47] L. Tang, Y. Dong, and J. Liu. 2015. Differential Evolution With an IndividualDependent Mechanism. IEEE TEVC 19, 4 (2015), 560–574.
[48] Y. Wang, Z. Cai, and Q. Zhang. 2011. Differential Evolution With Composite Trial
Vector Generation Strategies and Control Parameters. IEEE TEVC 15, 1 (2011),
55–66.
[49] Z. Yang, K. Tang, and X. Yao. 2008. Self-adaptive Differential Evolution with
Neighborhood Search. In IEEE CEC. 1110–1116.
[50] B. Yuan and M. Gallagher. 2007. Combining Meta-EAs and Racing for Difficult
EA Parameter Tuning Tasks. In Parameter Setting in Evolutionary Algorithms.
121–142.
[51] Z. Yuan, M. A. M. de Oca, M. Birattari, and T. Stützle. 2012. Continuous optimization algorithms for tuning real and integer parameters of swarm intelligencealgorithms. Swarm Intell. 6, 1 (2012), 49–75.
[52] A. Zamuda and J. Brest. 2015. Self-adaptive control parameters’ randomization
frequency and propagations in differential evolution. Swarm and Evol. Comput.
25 (2015), 72–99.
[53] J. Zhang and A. C. Sanderson. 2009. JADE: Adaptive Differential Evolution With
Optional External Archive. IEEE TEVC 13, 5 (2009), 945–958.
[54] K. Zielinski, X. Wang, and R. Laur. 2008. Comparison of Adaptive Approaches
for Differential Evolution. In PPSN. 641–650.
[55] K. Zielinski, P. Weitkemper, R. Laur, and K. D. Kammeyer. 2006. Parameter
Study for Differential Evolution Using a Power Allocation Problem IncludingInterference Cancellation. In IEEE CEC. 1857–1864.