Supplementary materials for “Reevaluating
Exponential Crossover in Differential Evolution”
Ryoji Tanabe and Alex Fukunaga
31/Mar/2014

1

8

10

7

10

6

10

5

1
0.8

10

Success rate

Avg. # of evaluations / SR

10exponential (A)exponential (D)binomial (D)SEC (D)

4102030 40 50 60
Dimension sizes

0.6
0.4
0.2

70

exponential (A)exponential (D)binomial (D)SEC (D)

0

80

10

10

8

10

7

10

6

10

5

40
50
60
Dimension sizes

70

80

exponential (A)exponential (D)binomial (D)SEC (D)

70

80

1
0.8
0.6
0.4
0.2

10

4

10

20

30 40 50 60
Dimension sizes

70

exponential (A)exponential (D)binomial (D)SEC (D)

08010(c) Schwefels 1.2 (Q)
10

7

10

6

10

5

20

30

40
50
60
Dimension sizes(d) Schwefels 1.2 (SR)

1

10

0.8
Success rate

Avg. # of evaluations / SR30(b) Rosenbrock (SR)

Success rate

Avg. # of evaluations / SR(a) Rosenbrock (Q)20exponential (A)exponential (D)binomial (D)SEC (D)

4102030 40 50 60
Dimension sizes

0.6
0.4
0.2

70

(e) Brock-rotated Ellipsoid (Q)80

exponential (A)exponential (D)binomial (D)SEC (D)

0
10

20

30

40
50
60
Dimension sizes

7080(f) Brock-rotated Ellipsoid (SR)

Figure 1: Evaluation of exponential crossover (standard DE). The horizontal
axis represents the dimensionality D, and the vertical axis represents: (left) themean fitness evaluations (for successful runs) divided by success rate, (right)
the success rate which is the number of “successes” divided by 50.

8

10

7

10

6

1

10

0.8
Success rate

Avg. # of evaluations / SR

10exponential (A)exponential (D)binomial (D)SEC (D)5102030 40 50 60
Dimension sizes

0.6
0.4
0.2

70

exponential (A)exponential (D)binomial (D)SEC (D)

0

80

10

10

7

10

6

10

5

40
50
60
Dimension sizes

70

80

70

80

1

10

0.8

exponential (A)exponential (D)binomial (D)SEC (D)

4102030 40 50 60
Dimension sizes

0.6
0.4
0.2

70

exponential (A)exponential (D)binomial (D)SEC (D)

08010(c) Schwefels 1.2 (Q)
10

7

10

6

10

5

20

30

40
50
60
Dimension sizes(d) Schwefels 1.2 (SR)

1

10

0.8
Success rate

Avg. # of evaluations / SR30(b) Rosenbrock (SR)

Success rate

Avg. # of evaluations / SR(a) Rosenbrock (Q)20exponential (A)exponential (D)binomial (D)SEC (D)

4102030 40 50 60
Dimension sizes

0.6
0.4
0.2

70

(e) Brock-rotated Ellipsoid (Q)80

exponential (A)exponential (D)binomial (D)SEC (D)

0
10

20

30

40
50
60
Dimension sizes

7080(f) Brock-rotated Ellipsoid (SR)

Figure 2: Evaluation of exponential crossover (jDE). The horizontal axis represents the dimensionality D, and the vertical axis represents: (left) the mean
fitness evaluations (for successful runs) divided by success rate, (right) the
success rate which is the number of “successes” divided by 50.

8

10

7

10

6

10

5

1
0.8

10

Success rate

Avg. # of evaluations / SR

10exponential (A)exponential (D)binomial (D)SEC (D)

4102030 40 50 60
Dimension sizes

0.6
0.4
0.2

70

exponential (A)exponential (D)binomial (D)SEC (D)

0

80

10

10

8

10

7

10

6

10

5

40
50
60
Dimension sizes

70

80

70

80

1
0.8

10

exponential (A)exponential (D)binomial (D)SEC (D)

4102030 40 50 60
Dimension sizes

0.6
0.4
0.2

70

exponential (A)exponential (D)binomial (D)SEC (D)

08010(c) Schwefels 1.2 (Q)
10

8

10

7

10

6

20

30

40
50
60
Dimension sizes(d) Schwefels 1.2 (SR)

1
0.8

10

10

Success rate

Avg. # of evaluations / SR30(b) Rosenbrock (SR)

Success rate

Avg. # of evaluations / SR(a) Rosenbrock (Q)20exponential (A)exponential (D)binomial (D)SEC (D)

5

4

10

20

30 40 50 60
Dimension sizes

0.6
0.4
0.2

70

(e) Brock-rotated Ellipsoid (Q)80

exponential (A)exponential (D)binomial (D)SEC (D)

0
10

20

30

40
50
60
Dimension sizes

7080(f) Brock-rotated Ellipsoid (SR)

Figure 3: Evaluation of exponential crossover (JADE). The horizontal axis
represents the dimensionality D, and the vertical axis represents: (left) themean fitness evaluations (for successful runs) divided by success rate, (right)
the success rate which is the number of “successes” divided by 50.

7

10

6

10

5

1

10

0.8
Success rate

Avg. # of evaluations / SR

10exponential (A)exponential (D)binomial (D)SEC (D)

4102030 40 50 60
Dimension sizes

0.6
0.4
0.2

70

exponential (A)exponential (D)binomial (D)SEC (D)

0

80

10

10

7

10

6

10

5

40
50
60
Dimension sizes

70

80

70

80

1

10

0.8

exponential (A)exponential (D)binomial (D)SEC (D)

4102030 40 50 60
Dimension sizes

0.6
0.4
0.2

70

exponential (A)exponential (D)binomial (D)SEC (D)

08010(c) Schwefels 1.2 (Q)
10

7

10

6

10

5

20

30

40
50
60
Dimension sizes(d) Schwefels 1.2 (SR)

1

10

0.8
Success rate

Avg. # of evaluations / SR30(b) Rosenbrock (SR)

Success rate

Avg. # of evaluations / SR(a) Rosenbrock (Q)20exponential (A)exponential (D)binomial (D)SEC (D)

4102030 40 50 60
Dimension sizes

0.6
0.4
0.2

70

(e) Brock-rotated Ellipsoid (Q)80

exponential (A)exponential (D)binomial (D)SEC (D)

0
10

20

30

40
50
60
Dimension sizes

7080(f) Brock-rotated Ellipsoid (SR)

Figure 4: Evaluation of exponential crossover (SHADE). The horizontal axis
represents the dimensionality D, and the vertical axis represents: (left) themean fitness evaluations (for successful runs) divided by success rate, (right)
the success rate which is the number of “successes” divided by 50.

Table 1: Comparison of shuffle exponential crossover with binomial and exponential crossover in standard DE on the CEC2014 benchmark functions [1](10 dimensions). For all problems, the maximum number of objective function
evaluations is D × 10, 000 = 100, 000. All results are the means of 51 runs.

F

shuffle exponential
Mean±Std Dev

exponential
Mean±Std Dev

binomial
Mean±Std Dev

F1
F2
F3
F4
F5
F6
F7
F8
F9
F10
F11
F12
F13
F14
F15
F16
F17
F18
F19
F20
F21
F22
F23
F24
F25
F26
F27
F28
F29
F30

0.00e+00±0.00e+00
0.00e+00±0.00e+00
0.00e+00±0.00e+00
6.48e+00±1.25e+01
1.97e+01±1.16e+00
0.00e+00±0.00e+00
3.75e-02±2.64e-02
0.00e+00±0.00e+00
1.18e+01±2.25e+00
4.19e+00±2.19e+00
5.54e+02±1.30e+02
4.71e-01±9.08e-02
1.25e-01±3.07e-02
1.43e-01±3.17e-02
1.36e+00±2.12e-01
2.31e+00±2.12e-01
3.86e-01±2.49e-01
1.56e-01±1.47e-01
2.94e-01±1.03e-01
4.77e-02±9.93e-02
2.27e-01±2.55e-01
1.16e-01±1.11e-01
3.23e+02±4.61e+01
1.15e+02±6.08e+00
1.47e+02±4.11e+01
1.00e+02±2.25e-02
6.06e+01±1.38e+02
3.66e+02±1.72e+01
2.03e+02±4.00e+01
4.62e+02±1.28e+00

0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
9.46e+00±1.43e+01≈
1.96e+01±3.00e+00+
0.00e+00±0.00e+00≈
3.70e-02±4.61e-02≈
0.00e+00±0.00e+00≈
1.18e+01±2.19e+00≈
4.58e+00±2.23e+00≈
5.98e+02±1.12e+02≈
4.67e-01±8.00e-02≈
1.26e-01±3.26e-02≈
1.34e-01±3.54e-02≈
1.32e+00±1.79e-01≈
2.33e+00±2.22e-01≈
3.58e-01±2.24e-01+
1.62e-01±2.92e-01−
3.21e-01±9.55e-02≈
7.42e-02±1.12e-01−
3.47e-01±2.69e-01−
1.97e-01±1.77e-01−
3.29e+02±0.00e+00≈
1.17e+02±6.54e+00−
1.44e+02±3.42e+01≈
1.00e+02±3.57e-02≈
5.07e+01±1.25e+02≈
3.63e+02±5.93e+00≈
2.04e+02±3.55e+01≈
4.63e+02±7.70e-01−

0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
7.58e+00±1.37e+01≈
1.91e+01±4.68e+00+
0.00e+00±0.00e+00≈
2.91e-02±2.14e-02≈
1.14e+01±7.39e+00−
1.42e+01±8.25e+00≈
5.19e+02±3.48e+02−
8.66e+02±3.03e+02−
6.55e-01±2.12e-01−
1.22e-01±2.64e-02≈
1.51e-01±3.02e-02≈
1.77e+00±4.66e-01−
2.51e+00±3.02e-01−
4.88e-01±1.53e+00−
1.43e-01±1.30e-01≈
3.26e-01±9.03e-02−
8.72e-02±1.44e-01≈
3.09e-01±2.55e-01≈
2.34e-01±2.11e-01−
3.29e+02±0.00e+00≈
1.13e+02±7.86e+00≈
1.42e+02±4.17e+01≈
1.00e+02±2.57e-02≈
4.68e+01±1.25e+02+
3.71e+02±2.83e+01≈
2.11e+02±3.11e+01−
4.64e+02±5.22e+00−

2
6
22

2
11
17

vs. shuffle exponential + (better)(Wilcoxon rank-sum) − (worse)p < 0.05 ≈ (no sig.)Table 2: Comparison of shuffle exponential crossover with binomial and exponential crossover in standard DE on the CEC2014 benchmark functions [1](30 dimensions). For all problems, the maximum number of objective function
evaluations is D × 10, 000 = 300, 000. All results are the means of 51 runs.

F

shuffle exponential
Mean±Std Dev

exponential
Mean±Std Dev

binomial
Mean±Std Dev

F1
F2
F3
F4
F5
F6
F7
F8
F9
F10
F11
F12
F13
F14
F15
F16
F17
F18
F19
F20
F21
F22
F23
F24
F25
F26
F27
F28
F29
F30

6.74e+04±6.78e+04
0.00e+00±0.00e+00
0.00e+00±0.00e+00
9.38e-02±1.01e-01
2.05e+01±3.99e-02
5.36e-01±7.07e-01
0.00e+00±0.00e+00
0.00e+00±0.00e+00
8.81e+01±9.61e+00
2.30e+01±4.92e+00
3.83e+03±2.45e+02
7.04e-01±1.02e-01
2.89e-01±8.40e-02
2.54e-01±3.20e-02
9.73e+00±8.17e-01
1.09e+01±3.29e-01
1.86e+02±1.71e+02
1.00e+01±4.63e+00
3.32e+00±1.09e+00
9.98e+00±6.40e+00
9.29e+01±1.02e+02
3.32e+01±2.42e+01
3.15e+02±0.00e+00
2.16e+02±9.54e+00
2.03e+02±1.38e-01
1.00e+02±7.29e-02
3.64e+02±4.85e+01
7.90e+02±2.06e+01
6.52e+02±1.56e+02
5.92e+02±1.72e+02

1.05e+05±8.05e+04−
0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
4.79e-01±3.63e-01−
2.05e+01±3.85e-02≈
3.85e-01±7.94e-01≈
0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
9.14e+01±9.01e+00≈
2.66e+01±4.31e+00−
3.83e+03±2.11e+02≈
7.35e-01±7.55e-02≈
3.15e-01±7.03e-02≈
2.54e-01±2.63e-02≈
9.60e+00±8.70e-01≈
1.09e+01±2.66e-01≈
1.83e+02±1.21e+02≈
1.12e+01±4.06e+00≈
3.40e+00±7.08e-01≈
9.39e+00±3.80e+00≈
7.98e+01±9.69e+01≈
3.65e+01±3.04e+01≈
3.15e+02±0.00e+00≈
2.15e+02±1.00e+01≈
2.03e+02±2.27e-01−
1.00e+02±7.76e-02≈
3.64e+02±4.59e+01≈
7.85e+02±3.20e+01+
6.63e+02±1.65e+02≈
6.27e+02±2.28e+02≈

3.29e+04±2.47e+04+
0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
2.06e-01±1.35e-01−
2.07e+01±1.87e-01−
3.74e-01±5.56e-01+
0.00e+00±0.00e+00≈
1.88e+01±5.59e+00−
3.60e+01±2.29e+01+
1.16e+03±1.43e+03−
3.89e+03±1.70e+03−
7.54e-01±6.35e-01−
2.77e-01±7.15e-02≈
2.59e-01±3.63e-02≈
6.64e+00±3.33e+00+
1.09e+01±1.34e+00≈
1.25e+02±1.08e+02≈
7.78e+00±3.52e+00+
3.08e+00±7.66e-01≈
7.58e+00±4.44e+00+
6.33e+01±8.86e+01+
2.81e+01±5.03e+00+
3.15e+02±0.00e+00≈
2.18e+02±8.97e+00≈
2.03e+02±1.51e-01≈
1.00e+02±7.07e-02≈
3.47e+02±4.78e+01+
8.00e+02±2.34e+01−
5.72e+02±2.45e+02+
5.25e+02±1.78e+02+

1
4
25

11
7
12

vs. shuffle exponential + (better)(Wilcoxon rank-sum) − (worse)p < 0.05 ≈ (no sig.)Table 3: Comparison of shuffle exponential crossover with binomial and exponential crossover in standard DE on the CEC2014 benchmark functions [1](50 dimensions). For all problems, the maximum number of objective function
evaluations is D × 10, 000 = 500, 000. All results are the means of 51 runs.

F

shuffle exponential
Mean±Std Dev

exponential
Mean±Std Dev

binomial
Mean±Std Dev

F1
F2
F3
F4
F5
F6
F7
F8
F9
F10
F11
F12
F13
F14
F15
F16
F17
F18
F19
F20
F21
F22
F23
F24
F25
F26
F27
F28
F29
F30

5.62e+05±2.49e+05
9.65e+00±1.60e+01
1.30e-05±1.14e-05
7.50e+01±3.26e+01
2.06e+01±3.03e-02
2.40e+00±1.93e+00
0.00e+00±0.00e+00
0.00e+00±0.00e+00
1.97e+02±1.33e+01
3.49e+01±5.71e+00
7.61e+03±2.96e+02
8.13e-01±8.02e-02
4.30e-01±8.04e-02
2.75e-01±3.65e-02
2.14e+01±1.37e+00
1.96e+01±3.18e-01
1.46e+04±9.23e+03
4.40e+01±1.87e+01
8.15e+00±2.06e+00
4.55e+01±2.89e+01
1.02e+03±8.02e+02
5.09e+02±2.65e+02
3.44e+02±0.00e+00
2.62e+02±3.65e+00
2.06e+02±6.71e-01
1.00e+02±4.37e-02
3.56e+02±3.49e+01
1.08e+03±5.54e+01
8.87e+02±6.75e+01
7.93e+03±9.66e+01

6.57e+05±3.11e+05≈
6.73e+02±8.56e+02−
3.12e-04±2.65e-04−
8.40e+01±2.52e+01≈
2.06e+01±3.01e-02≈
3.05e+00±2.26e+00≈
0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
2.00e+02±1.14e+01≈
4.28e+01±6.64e+00−
7.69e+03±2.68e+02≈
8.10e-01±7.04e-02≈
4.42e-01±7.68e-02≈
2.81e-01±3.34e-02≈
2.10e+01±1.51e+00≈
1.97e+01±3.00e-01≈
1.65e+04±1.51e+04≈
4.93e+01±1.80e+01≈
8.61e+00±1.97e+00≈
5.83e+01±2.43e+01−
1.18e+03±1.19e+03≈
5.64e+02±2.10e+02≈
3.44e+02±0.00e+00≈
2.63e+02±3.76e+00≈
2.06e+02±5.72e-01≈
1.00e+02±1.01e-01+
3.74e+02±2.75e+01−
1.10e+03±4.72e+01−
1.15e+03±2.49e+02−
7.98e+03±1.27e+02−

3.85e+05±1.78e+05+
2.08e+02±6.43e+02≈
1.93e-01±3.55e-01−
7.06e+01±2.99e+01≈
2.08e+01±1.27e-01−
1.50e+00±1.67e+00+
1.45e-04±1.04e-03≈
3.46e+01±7.07e+00−
5.85e+01±1.45e+01+
1.60e+03±1.22e+03−
6.98e+03±3.04e+03+
5.46e-01±4.20e-01+
3.96e-01±7.35e-02+
3.17e-01±6.58e-02−
9.11e+00±4.66e+00+
1.93e+01±1.12e+00+
1.25e+04±6.44e+03≈
2.35e+01±8.63e+00+
9.12e+00±2.35e+00−
4.90e+01±3.37e+01≈
7.16e+02±4.44e+02+
5.91e+02±3.61e+02≈
3.44e+02±0.00e+00≈
2.70e+02±2.36e+00−
2.05e+02±4.10e-01+
1.00e+02±7.98e-02+
3.69e+02±3.80e+01−
1.08e+03±3.65e+01≈
9.13e+02±2.15e+02≈
8.27e+03±3.22e+02−

1
8
21

12
9
9

vs. shuffle exponential + (better)(Wilcoxon rank-sum) − (worse)p < 0.05 ≈ (no sig.)Table 4: Comparison of shuffle exponential crossover with binomial and exponential crossover in jDE on the CEC2014 benchmark functions [1] (10 dimensions). For all problems, the maximum number of objective function evaluations
is D × 10, 000 = 100, 000. All results are the means of 51 runs.

F

shuffle exponential
Mean±Std Dev

exponential
Mean±Std Dev

binomial
Mean±Std Dev

F1
F2
F3
F4
F5
F6
F7
F8
F9
F10
F11
F12
F13
F14
F15
F16
F17
F18
F19
F20
F21
F22
F23
F24
F25
F26
F27
F28
F29
F30

1.45e-05±2.46e-05
9.28e-07±1.99e-06
1.70e-07±5.00e-07
3.93e+00±1.04e+01
1.86e+01±3.49e+00
1.52e-01±1.74e-01
2.16e-02±1.11e-02
0.00e+00±0.00e+00
5.67e+00±1.21e+00
0.00e+00±0.00e+00
2.67e+02±8.70e+01
3.31e-01±6.11e-02
1.64e-01±2.62e-02
1.60e-01±3.35e-02
9.40e-01±1.66e-01
2.08e+00±2.70e-01
4.81e+01±2.31e+01
3.22e+00±1.04e+00
3.49e-01±1.09e-01
4.80e-01±1.92e-01
3.88e+00±2.69e+00
3.58e-01±4.19e-01
3.29e+02±0.00e+00
1.13e+02±2.19e+00
1.27e+02±1.44e+01
1.00e+02±2.62e-02
4.68e+01±1.12e+02
3.57e+02±1.76e+00
2.23e+02±7.81e-01
4.73e+02±1.17e+01

1.09e-03±4.60e-03≈
3.21e-05±5.32e-05−
2.54e-09±7.04e-09+
1.29e+00±5.01e+00≈
1.81e+01±3.60e+00≈
1.07e-02±2.69e-02+
1.90e-02±1.09e-02≈
0.00e+00±0.00e+00≈
5.62e+00±1.23e+00≈
0.00e+00±0.00e+00≈
2.48e+02±1.09e+02≈
3.37e-01±6.28e-02≈
1.61e-01±2.96e-02≈
1.66e-01±3.55e-02≈
8.92e-01±1.52e-01≈
2.08e+00±2.50e-01≈
4.87e+01±2.80e+01≈
4.16e+00±1.32e+00−
3.78e-01±1.18e-01≈
1.07e+00±4.40e-01−
1.04e+01±7.82e+00−
7.02e-01±5.70e-01−
3.29e+02±0.00e+00≈
1.13e+02±1.89e+00≈
1.31e+02±2.18e+01≈
1.00e+02±2.81e-02≈
6.18e+01±1.21e+02≈
3.58e+02±4.62e+00−
2.24e+02±6.59e-01≈
4.77e+02±1.39e+01−

0.00e+00±0.00e+00+
0.00e+00±0.00e+00+
0.00e+00±0.00e+00+
1.42e+01±1.67e+01≈
1.98e+01±1.30e+00−
3.25e-06±1.70e-05+
1.89e-02±1.38e-02≈
0.00e+00±0.00e+00≈
5.99e+00±1.25e+00≈
0.00e+00±0.00e+00≈
2.96e+02±9.12e+01≈
3.87e-01±6.99e-02−
1.42e-01±2.83e-02+
1.56e-01±4.28e-02≈
9.94e-01±1.78e-01≈
2.12e+00±2.66e-01≈
1.15e+01±1.03e+01+
1.37e+00±6.87e-01+
3.32e-01±1.05e-01≈
2.06e-01±1.07e-01+
4.99e-01±3.93e-01+
1.72e-01±9.86e-02+
3.29e+02±0.00e+00≈
1.13e+02±2.29e+00≈
1.27e+02±2.82e+01−
1.00e+02±3.17e-02+
7.94e+01±1.42e+02−
3.61e+02±6.03e+00≈
2.22e+02±5.15e-01+
4.66e+02±9.19e+00+

2
7
21

13
4
13

vs. shuffle exponential + (better)(Wilcoxon rank-sum) − (worse)p < 0.05 ≈ (no sig.)Table 5: Comparison of shuffle exponential crossover with binomial and exponential crossover in jDE on the CEC2014 benchmark functions [1] (30 dimensions). For all problems, the maximum number of objective function evaluations
is D × 10, 000 = 300, 000. All results are the means of 51 runs.

F

shuffle exponential
Mean±Std Dev

exponential
Mean±Std Dev

binomial
Mean±Std Dev

F1
F2
F3
F4
F5
F6
F7
F8
F9
F10
F11
F12
F13
F14
F15
F16
F17
F18
F19
F20
F21
F22
F23
F24
F25
F26
F27
F28
F29
F30

3.17e+05±1.91e+05
0.00e+00±0.00e+00
7.75e-06±1.25e-05
5.74e+01±2.87e+01
2.03e+01±3.00e-02
1.36e+01±9.86e-01
0.00e+00±0.00e+00
0.00e+00±0.00e+00
5.05e+01±6.47e+00
4.08e-04±2.92e-03
2.25e+03±2.36e+02
3.87e-01±4.13e-02
3.46e-01±3.25e-02
2.64e-01±2.81e-02
6.97e+00±5.98e-01
9.75e+00±3.88e-01
5.30e+04±5.19e+04
6.66e+02±6.64e+02
6.94e+00±5.47e-01
5.40e+01±1.40e+01
3.69e+03±2.02e+03
2.31e+02±6.96e+01
3.15e+02±0.00e+00
2.26e+02±6.35e-01
2.04e+02±5.70e-01
1.00e+02±3.90e-02
4.08e+02±5.20e+00
8.53e+02±1.67e+01
1.30e+03±9.66e+01
1.78e+03±4.68e+02

3.75e+05±2.30e+05≈
3.38e-09±1.58e-08−
7.25e-05±1.04e-04−
5.35e+01±2.93e+01≈
2.03e+01±3.42e-02≈
1.31e+01±1.11e+00+
3.67e-10±2.62e-09≈
0.00e+00±0.00e+00≈
5.09e+01±6.67e+00≈
8.16e-04±4.08e-03≈
2.26e+03±2.71e+02≈
3.91e-01±4.20e-02≈
3.47e-01±3.93e-02≈
2.66e-01±3.00e-02≈
6.77e+00±8.12e-01≈
9.79e+00±2.69e-01≈
5.65e+04±4.53e+04≈
7.03e+02±6.86e+02≈
6.97e+00±6.14e-01≈
5.12e+01±1.31e+01≈
3.20e+03±1.71e+03≈
2.13e+02±8.10e+01≈
3.15e+02±0.00e+00≈
2.26e+02±6.79e-01≈
2.04e+02±6.14e-01≈
1.00e+02±4.51e-02≈
4.11e+02±8.88e+00−
8.53e+02±1.99e+01≈
1.30e+03±1.06e+02≈
1.61e+03±4.34e+02+

6.54e+04±4.95e+04+
0.00e+00±0.00e+00≈
0.00e+00±0.00e+00+
5.08e+00±1.60e+01+
2.03e+01±4.29e-02−
3.84e+00±4.37e+00+
0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
4.25e+01±6.64e+00+
2.04e-03±6.25e-03≈
2.43e+03±2.68e+02−
4.63e-01±5.13e-02−
2.97e-01±3.60e-02+
2.76e-01±3.22e-02−
5.66e+00±7.18e-01+
9.89e+00±3.65e-01≈
1.61e+03±1.46e+03+
1.78e+01±1.19e+01+
4.49e+00±6.49e-01+
1.16e+01±3.15e+00+
3.03e+02±1.36e+02+
9.91e+01±5.98e+01+
3.15e+02±0.00e+00≈
2.24e+02±1.46e+00+
2.03e+02±6.34e-01+
1.00e+02±3.67e-02+
3.56e+02±4.86e+01+
7.88e+02±3.10e+01+
8.07e+02±7.49e+01+
1.49e+03±6.80e+02+

2
3
25

20
4
6

vs. shuffle exponential + (better)(Wilcoxon rank-sum) − (worse)p < 0.05 ≈ (no sig.)Table 6: Comparison of shuffle exponential crossover with binomial and exponential crossover in jDE on the CEC2014 benchmark functions [1] (50 dimensions). For all problems, the maximum number of objective function evaluations
is D × 10, 000 = 500, 000. All results are the means of 51 runs.

F

shuffle exponential
Mean±Std Dev

exponential
Mean±Std Dev

binomial
Mean±Std Dev

F1
F2
F3
F4
F5
F6
F7
F8
F9
F10
F11
F12
F13
F14
F15
F16
F17
F18
F19
F20
F21
F22
F23
F24
F25
F26
F27
F28
F29
F30

1.34e+06±6.59e+05
1.49e+03±9.54e+02
5.28e+01±4.38e+01
9.48e+01±4.31e+00
2.04e+01±2.90e-02
2.87e+01±1.83e+00
1.82e-05±4.91e-05
0.00e+00±0.00e+00
1.22e+02±9.66e+00
9.80e-04±3.39e-03
4.95e+03±3.04e+02
4.17e-01±4.30e-02
4.05e-01±4.22e-02
3.14e-01±1.83e-02
1.56e+01±1.51e+00
1.82e+01±3.54e-01
7.26e+05±5.12e+05
9.80e+02±5.90e+02
1.78e+01±2.02e+00
6.85e+02±3.46e+02
1.28e+05±6.78e+04
7.53e+02±1.33e+02
3.44e+02±0.00e+00
2.61e+02±3.44e+00
2.11e+02±1.42e+00
1.00e+02±3.89e-02
9.69e+02±1.84e+02
1.29e+03±3.44e+01
1.66e+03±2.05e+02
8.64e+03±2.51e+02

1.38e+06±6.40e+05≈
1.63e+03±1.40e+03≈
3.94e+01±3.44e+01≈
9.12e+01±1.68e+01≈
2.04e+01±2.76e-02≈
2.90e+01±1.67e+00≈
8.07e-05±1.53e-04−
0.00e+00±0.00e+00≈
1.24e+02±1.21e+01≈
1.22e-03±3.75e-03≈
4.90e+03±3.01e+02≈
4.09e-01±5.17e-02≈
3.99e-01±3.77e-02≈
3.13e-01±2.23e-02≈
1.56e+01±1.67e+00≈
1.82e+01±3.30e-01≈
6.73e+05±4.09e+05≈
1.08e+03±6.24e+02≈
1.78e+01±2.64e+00≈
9.12e+02±4.63e+02−
1.28e+05±6.10e+04≈
7.29e+02±1.63e+02≈
3.44e+02±0.00e+00≈
2.62e+02±3.64e+00≈
2.12e+02±1.49e+00≈
1.00e+02±4.59e-02≈
1.03e+03±9.84e+01≈
1.29e+03±3.67e+01≈
1.70e+03±2.13e+02≈
8.66e+03±2.57e+02≈

4.91e+05±2.37e+05+
8.17e-09±3.61e-08+
7.18e-09±2.27e-08+
8.91e+01±1.26e+01≈
2.04e+01±2.99e-02−
8.33e+00±6.88e+00+
0.00e+00±0.00e+00+
0.00e+00±0.00e+00≈
9.34e+01±9.11e+00+
3.43e-03±6.65e-03−
5.21e+03±3.70e+02−
4.94e-01±5.80e-02−
3.90e-01±4.56e-02≈
3.40e-01±9.84e-02≈
1.21e+01±1.15e+00+
1.82e+01±4.21e-01≈
2.15e+04±1.43e+04+
4.56e+02±5.59e+02+
1.43e+01±6.16e+00+
5.17e+01±2.18e+01+
8.36e+03±7.71e+03+
5.46e+02±1.52e+02+
3.44e+02±0.00e+00≈
2.68e+02±2.07e+00−
2.07e+02±1.97e+00+
1.00e+02±3.52e-02+
4.33e+02±5.33e+01+
1.11e+03±3.62e+01+
1.08e+03±1.95e+02+
8.65e+03±4.68e+02≈

0
2
28

18
5
7

vs. shuffle exponential + (better)(Wilcoxon rank-sum) − (worse)p < 0.05 ≈ (no sig.)Table 7: Comparison of shuffle exponential crossover with binomial and exponential crossover in JADE on the CEC2014 benchmark functions [1] (10 dimensions). For all problems, the maximum number of objective function evaluations
is D × 10, 000 = 100, 000. All results are the means of 51 runs.

F

shuffle exponential
Mean±Std Dev

exponential
Mean±Std Dev

binomial
Mean±Std Dev

F1
F2
F3
F4
F5
F6
F7
F8
F9
F10
F11
F12
F13
F14
F15
F16
F17
F18
F19
F20
F21
F22
F23
F24
F25
F26
F27
F28
F29
F30

2.78e+04±1.96e+04
5.94e+01±5.06e+01
3.71e+01±5.31e+01
2.32e+00±6.57e+00
1.74e+01±4.82e+00
1.15e+00±3.25e-01
1.87e-02±6.88e-03
0.00e+00±0.00e+00
4.13e+00±9.57e-01
2.45e-03±1.22e-02
1.17e+02±5.99e+01
2.34e-01±4.96e-02
1.33e-01±2.63e-02
1.13e-01±3.19e-02
6.71e-01±1.36e-01
1.82e+00±3.25e-01
7.37e+03±6.39e+03
1.92e+02±2.47e+02
2.69e-01±7.46e-02
9.93e-01±6.15e-01
7.03e+02±6.99e+02
1.93e-01±1.03e-01
3.04e+02±8.78e+01
1.11e+02±2.03e+00
1.26e+02±5.55e+00
1.00e+02±3.07e-02
4.94e+01±1.13e+02
3.68e+02±2.76e+01
3.12e+02±3.49e+01
5.43e+02±3.44e+01

7.13e+04±3.24e+04−
9.72e+01±9.41e+01≈
2.53e+01±2.38e+01≈
1.32e+00±3.39e+00≈
1.77e+01±4.45e+00≈
4.86e-01±1.61e-01+
2.01e-02±8.58e-03≈
0.00e+00±0.00e+00≈
4.21e+00±8.75e-01≈
4.90e-03±1.70e-02≈
1.34e+02±8.31e+01≈
2.38e-01±5.18e-02≈
1.29e-01±2.47e-02≈
1.14e-01±3.60e-02≈
6.38e-01±1.26e-01≈
1.85e+00±1.99e-01≈
1.10e+04±1.03e+04≈
5.38e+02±4.43e+02−
3.05e-01±9.07e-02−
8.75e+01±1.04e+02−
8.54e+02±8.34e+02≈
2.35e-01±8.86e-02−
3.17e+02±6.17e+01−
1.13e+02±1.89e+00−
1.28e+02±1.11e+01≈
1.00e+02±3.17e-02≈
4.28e+01±1.06e+02≈
3.74e+02±3.46e+01−
3.19e+02±3.91e+01≈
5.58e+02±4.32e+01≈

0.00e+00±0.00e+00+
0.00e+00±0.00e+00+
3.52e-05±2.07e-04+
2.34e+01±1.62e+01−
1.77e+01±4.42e+00≈
9.92e-02±1.34e-01+
1.05e-02±5.37e-03+
0.00e+00±0.00e+00≈
3.29e+00±8.73e-01+
9.80e-03±2.29e-02−
8.83e+01±5.47e+01+
2.55e-01±5.56e-02≈
8.16e-02±1.98e-02+
1.02e-01±3.64e-02≈
5.54e-01±9.35e-02+
1.63e+00±2.93e-01+
3.14e+00±1.71e+01+
4.00e-01±5.10e-01+
2.71e-01±1.72e-01≈
3.25e-01±8.75e-02+
8.45e-01±1.53e+00+
1.87e-01±8.77e-02≈
3.29e+02±0.00e+00−
1.09e+02±1.71e+00+
1.25e+02±2.30e+01+
1.00e+02±1.45e-02+
6.55e+01±1.39e+02−
4.03e+02±5.10e+01−
2.33e+02±2.83e+01+
4.76e+02±2.13e+01+

1
8
21

19
5
6

vs. shuffle exponential + (better)(Wilcoxon rank-sum) − (worse)p < 0.05 ≈ (no sig.)Table 8: Comparison of shuffle exponential crossover with binomial and exponential crossover in JADE on the CEC2014 benchmark functions [1] (30 dimensions). For all problems, the maximum number of objective function evaluations
is D × 10, 000 = 300, 000. All results are the means of 51 runs.

F

shuffle exponential
Mean±Std Dev

exponential
Mean±Std Dev

binomial
Mean±Std Dev

F1
F2
F3
F4
F5
F6
F7
F8
F9
F10
F11
F12
F13
F14
F15
F16
F17
F18
F19
F20
F21
F22
F23
F24
F25
F26
F27
F28
F29
F30

1.68e+07±5.71e+06
5.37e+01±5.32e+01
4.28e+02±5.30e+02
5.99e+01±2.17e+01
2.02e+01±2.92e-02
1.33e+01±1.11e+00
1.53e-04±1.07e-04
0.00e+00±0.00e+00
4.29e+01±5.73e+00
2.45e-03±6.77e-03
1.65e+03±2.05e+02
2.27e-01±2.96e-02
2.94e-01±3.83e-02
2.16e-01±2.26e-02
5.29e+00±5.84e-01
9.51e+00±3.28e-01
2.54e+06±1.18e+06
1.62e+04±9.81e+03
7.21e+00±7.23e-01
5.28e+03±2.16e+03
4.33e+05±2.30e+05
2.07e+02±8.65e+01
3.15e+02±4.42e-02
2.28e+02±7.51e-01
2.08e+02±8.08e-01
1.00e+02±5.38e-02
4.04e+02±5.69e+01
8.71e+02±3.48e+01
1.71e+03±1.32e+02
4.10e+03±8.39e+02

1.51e+07±4.72e+06≈
2.43e+02±2.61e+02−
6.94e+02±8.97e+02≈
5.10e+01±2.30e+01+
2.02e+01±3.83e-02≈
1.30e+01±1.08e+00≈
2.24e-04±1.86e-04−
0.00e+00±0.00e+00≈
4.41e+01±5.70e+00≈
2.45e-03±6.77e-03≈
1.70e+03±2.20e+02≈
2.19e-01±3.50e-02≈
3.00e-01±4.07e-02≈
2.20e-01±2.09e-02≈
5.14e+00±6.60e-01≈
9.53e+00±3.79e-01≈
2.83e+06±1.35e+06≈
1.69e+04±9.15e+03≈
7.20e+00±8.24e-01≈
5.44e+03±2.31e+03≈
5.03e+05±1.90e+05−
2.09e+02±6.98e+01≈
3.15e+02±6.44e-02≈
2.28e+02±6.55e-01≈
2.08e+02±1.06e+00−
1.00e+02±4.77e-02≈
4.13e+02±4.06e+00≈
8.68e+02±2.82e+01≈
1.67e+03±1.45e+02≈
4.13e+03±1.11e+03≈

1.81e+03±2.42e+03+
0.00e+00±0.00e+00+
8.40e-05±5.03e-04+
7.82e-02±5.58e-01+
2.03e+01±3.10e-02−
9.01e+00±2.71e+00+
5.32e-04±2.22e-03−
0.00e+00±0.00e+00≈
2.56e+01±4.47e+00+
6.94e-03±1.15e-02−
1.67e+03±2.16e+02≈
2.52e-01±3.60e-02−
2.05e-01±2.90e-02+
2.24e-01±3.48e-02≈
3.12e+00±3.91e-01+
9.39e+00±3.07e-01≈
1.12e+03±3.69e+02+
9.06e+01±1.17e+02+
4.47e+00±7.11e-01+
2.38e+03±2.28e+03+
1.07e+04±4.42e+04+
1.60e+02±6.56e+01+
3.15e+02±0.00e+00+
2.26e+02±3.90e+00+
2.05e+02±2.31e+00+
1.02e+02±1.40e+01−
3.42e+02±4.97e+01+
7.93e+02±4.17e+01+
7.28e+02±1.50e+01+
1.48e+03±4.78e+02+

1
4
25

21
5
4

vs. shuffle exponential + (better)(Wilcoxon rank-sum) − (worse)p < 0.05 ≈ (no sig.)Table 9: Comparison of shuffle exponential crossover with binomial and exponential crossover in JADE on the CEC2014 benchmark functions [1] (50 dimensions). For all problems, the maximum number of objective function evaluations
is D × 10, 000 = 500, 000. All results are the means of 51 runs.

F

shuffle exponential
Mean±Std Dev

exponential
Mean±Std Dev

binomial
Mean±Std Dev

F1
F2
F3
F4
F5
F6
F7
F8
F9
F10
F11
F12
F13
F14
F15
F16
F17
F18
F19
F20
F21
F22
F23
F24
F25
F26
F27
F28
F29
F30

1.60e+07±3.46e+06
4.19e+04±1.70e+04
5.81e+03±2.99e+03
8.53e+01±1.59e+01
2.03e+01±3.42e-02
2.58e+01±1.88e+00
5.49e-03±2.20e-03
0.00e+00±0.00e+00
1.00e+02±9.96e+00
3.43e-03±6.65e-03
3.82e+03±2.88e+02
2.07e-01±2.34e-02
3.71e-01±4.09e-02
2.72e-01±2.04e-02
1.25e+01±1.23e+00
1.78e+01±3.44e-01
6.64e+06±1.79e+06
1.41e+04±5.86e+03
1.80e+01±2.30e+00
2.16e+04±5.63e+03
3.38e+06±1.44e+06
5.79e+02±1.72e+02
3.45e+02±8.14e-01
2.58e+02±7.57e-01
2.16e+02±1.46e+00
1.00e+02±4.12e-02
8.77e+02±2.52e+02
1.42e+03±5.76e+01
2.66e+03±4.30e+02
1.07e+04±9.80e+02

1.73e+07±3.95e+06≈
4.26e+04±2.45e+04≈
6.68e+03±2.47e+03≈
8.69e+01±1.43e+01≈
2.03e+01±2.98e-02≈
2.56e+01±1.87e+00≈
6.06e-03±2.81e-03≈
0.00e+00±0.00e+00≈
1.02e+02±1.06e+01≈
6.61e-03±9.13e-03≈
3.83e+03±3.18e+02≈
2.05e-01±2.33e-02≈
3.73e-01±3.61e-02≈
2.72e-01±2.45e-02≈
1.25e+01±1.13e+00≈
1.79e+01±3.09e-01≈
6.55e+06±1.69e+06≈
1.41e+04±6.93e+03≈
1.86e+01±2.94e+00≈
2.16e+04±6.10e+03≈
3.57e+06±1.32e+06≈
5.69e+02±1.41e+02≈
3.46e+02±1.48e+00−
2.58e+02±5.80e-01≈
2.16e+02±1.34e+00−
1.00e+02±4.93e-02−
8.59e+02±2.70e+02≈
1.43e+03±6.32e+01≈
2.61e+03±3.46e+02≈
1.05e+04±8.04e+02≈

1.86e+04±1.45e+04+
0.00e+00±0.00e+00+
3.66e+03±2.17e+03+
5.47e+00±2.06e+01+
2.04e+01±3.42e-02−
1.67e+01±6.39e+00+
4.88e-03±5.94e-03≈
0.00e+00±0.00e+00≈
5.41e+01±7.49e+00+
8.57e-03±8.47e-03−
3.82e+03±3.11e+02≈
2.61e-01±2.75e-02−
3.17e-01±4.59e-02+
2.95e-01±6.33e-02−
7.48e+00±9.07e-01+
1.77e+01±4.16e-01≈
2.47e+03±6.84e+02+
1.63e+02±4.75e+01+
1.55e+01±7.24e+00+
6.23e+03±6.29e+03+
4.63e+04±2.30e+05+
4.95e+02±1.53e+02+
3.44e+02±0.00e+00+
2.74e+02±2.59e+00−
2.22e+02±3.82e+00−
1.02e+02±1.40e+01−
4.55e+02±5.63e+01+
1.14e+03±5.45e+01+
9.04e+02±8.26e+01+
1.00e+04±9.22e+02+

0
3
27

19
7
4

vs. shuffle exponential + (better)(Wilcoxon rank-sum) − (worse)p < 0.05 ≈ (no sig.)Table 10: Comparison of shuffle exponential crossover with binomial and exponential crossover in SHADE on the CEC2014 benchmark functions [1] (10dimensions). For all problems, the maximum number of objective function evaluations is D × 10, 000 = 100, 000. All results are the means of 51 runs.

F

shuffle exponential
Mean±Std Dev

exponential
Mean±Std Dev

binomial
Mean±Std Dev

F1
F2
F3
F4
F5
F6
F7
F8
F9
F10
F11
F12
F13
F14
F15
F16
F17
F18
F19
F20
F21
F22
F23
F24
F25
F26
F27
F28
F29
F30

0.00e+00±0.00e+00
0.00e+00±0.00e+00
0.00e+00±0.00e+00
1.34e+01±1.67e+01
1.63e+01±6.64e+00
1.58e-01±3.07e-01
4.89e-03±5.31e-03
0.00e+00±0.00e+00
3.40e+00±1.15e+00
2.45e-03±1.22e-02
9.26e+01±7.14e+01
1.54e-01±3.19e-02
1.20e-01±2.47e-02
9.97e-02±2.56e-02
5.31e-01±9.03e-02
1.86e+00±2.35e-01
1.34e+00±2.32e+00
1.79e-01±2.35e-01
1.92e-01±4.92e-02
1.91e-01±7.01e-02
4.09e-01±2.74e-01
1.58e-01±6.70e-02
3.29e+02±0.00e+00
1.10e+02±1.91e+00
1.16e+02±2.23e+01
1.00e+02±2.31e-02
4.70e+01±1.25e+02
3.72e+02±2.44e+01
2.22e+02±6.04e-01
4.66e+02±1.35e+01

0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
1.67e+01±1.72e+01≈
1.75e+01±5.46e+00≈
2.64e-03±6.68e-03+
4.72e-03±6.25e-03≈
0.00e+00±0.00e+00≈
3.45e+00±1.16e+00≈
1.22e-03±8.75e-03≈
9.62e+01±7.10e+01≈
1.56e-01±3.22e-02≈
1.21e-01±2.74e-02≈
1.00e-01±2.49e-02≈
5.00e-01±9.39e-02≈
1.86e+00±2.36e-01≈
1.26e+00±2.39e+00≈
3.71e-01±4.07e-01−
2.37e-01±1.44e-01≈
4.03e-01±1.73e-01−
4.04e-01±2.69e-01≈
2.61e-01±1.28e-01−
3.23e+02±4.61e+01≈
1.11e+02±2.01e+00≈
1.17e+02±1.77e+01≈
1.00e+02±2.50e-02≈
8.02e+01±1.60e+02≈
3.79e+02±3.36e+01≈
2.22e+02±5.73e-01≈
4.64e+02±6.82e+00≈

0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
2.87e+01±1.32e+01−
1.54e+01±7.08e+00≈
0.00e+00±0.00e+00+
7.75e-03±1.16e-02≈
0.00e+00±0.00e+00≈
2.92e+00±8.18e-01+
1.47e-02±2.95e-02−
7.57e+01±5.16e+01≈
1.42e-01±3.80e-02≈
7.15e-02±1.80e-02+
9.95e-02±3.09e-02≈
4.92e-01±8.10e-02≈
1.52e+00±2.75e-01+
1.66e+00±3.18e+00≈
2.67e-01±3.38e-01≈
2.16e-01±2.44e-01−
2.04e-01±1.59e-01≈
3.10e-01±2.48e-01≈
2.57e-01±8.25e-02−
3.29e+02±0.00e+00≈
1.09e+02±1.86e+00+
1.22e+02±2.74e+01−
1.00e+02±1.93e-02+
1.07e+02±1.60e+02−
3.97e+02±4.58e+01−
2.22e+02±5.07e-01≈
4.71e+02±2.12e+01≈

1
3
26

6
7
17

vs. shuffle exponential + (better)(Wilcoxon rank-sum) − (worse)p < 0.05 ≈ (no sig.)Table 11: Comparison of shuffle exponential crossover with binomial and exponential crossover in SHADE on the CEC2014 benchmark functions [1] (30dimensions). For all problems, the maximum number of objective function evaluations is D × 10, 000 = 300, 000. All results are the means of 51 runs.

F

shuffle exponential
Mean±Std Dev

exponential
Mean±Std Dev

binomial
Mean±Std Dev

F1
F2
F3
F4
F5
F6
F7
F8
F9
F10
F11
F12
F13
F14
F15
F16
F17
F18
F19
F20
F21
F22
F23
F24
F25
F26
F27
F28
F29
F30

1.32e+03±2.15e+03
0.00e+00±0.00e+00
0.00e+00±0.00e+00
1.24e+00±8.88e+00
2.01e+01±2.44e-02
1.19e+01±1.22e+00
0.00e+00±0.00e+00
0.00e+00±0.00e+00
4.11e+01±5.39e+00
5.31e-03±1.01e-02
1.70e+03±2.35e+02
1.79e-01±2.85e-02
3.02e-01±3.98e-02
2.13e-01±2.32e-02
3.15e+00±3.55e-01
9.72e+00±3.72e-01
8.63e+02±2.91e+02
3.14e+01±2.25e+01
4.06e+00±5.09e-01
1.32e+01±5.04e+00
2.01e+02±9.83e+01
1.06e+02±6.88e+01
3.15e+02±0.00e+00
2.25e+02±1.01e+00
2.03e+02±3.43e-01
1.00e+02±4.11e-02
4.04e+02±5.26e+00
8.14e+02±1.57e+01
7.26e+02±1.91e+01
1.26e+03±4.48e+02

1.62e+03±2.34e+03≈
0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
2.01e+01±1.91e-02≈
1.12e+01±1.15e+00+
0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
4.00e+01±5.15e+00≈
4.90e-03±9.84e-03≈
1.75e+03±1.75e+02≈
1.83e-01±2.28e-02≈
3.05e-01±4.19e-02≈
2.10e-01±2.61e-02≈
3.20e+00±4.11e-01≈
9.66e+00±3.44e-01≈
1.02e+03±3.33e+02−
2.85e+01±2.23e+01≈
4.15e+00±5.99e-01≈
1.36e+01±5.09e+00≈
2.35e+02±9.67e+01≈
1.28e+02±6.75e+01≈
3.15e+02±0.00e+00≈
2.24e+02±1.56e+00+
2.03e+02±4.07e-01≈
1.00e+02±5.21e-02≈
4.06e+02±4.33e+00−
8.21e+02±1.75e+01≈
7.23e+02±5.29e+01≈
1.17e+03±3.35e+02≈

8.53e+02±1.85e+03+
0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
2.01e+01±1.39e-02+
5.82e-01±7.23e-01+
1.45e-04±1.04e-03≈
0.00e+00±0.00e+00≈
1.51e+01±2.54e+00+
1.02e-02±1.58e-02−
1.44e+03±2.20e+02+
1.61e-01±2.19e-02+
2.16e-01±3.26e-02+
2.39e-01±3.56e-02−
2.53e+00±2.74e-01+
9.05e+00±4.26e-01+
1.02e+03±3.40e+02−
5.54e+01±3.01e+01−
4.37e+00±6.91e-01−
1.36e+01±8.45e+00≈
2.39e+02±1.28e+02≈
1.08e+02±6.69e+01≈
3.15e+02±0.00e+00≈
2.26e+02±3.38e+00≈
2.04e+02±8.06e-01−
1.02e+02±1.40e+01−
3.23e+02±3.63e+01+
8.28e+02±3.87e+01−
7.16e+02±3.92e+01≈
1.56e+03±7.59e+02≈

2
2
26

10
8
12

vs. shuffle exponential + (better)(Wilcoxon rank-sum) − (worse)p < 0.05 ≈ (no sig.)Table 12: Comparison of shuffle exponential crossover with binomial and exponential crossover in SHADE on the CEC2014 benchmark functions [1] (50dimensions). For all problems, the maximum number of objective function evaluations is D × 10, 000 = 500, 000. All results are the means of 51 runs.

F

shuffle exponential
Mean±Std Dev

exponential
Mean±Std Dev

binomial
Mean±Std Dev

F1
F2
F3
F4
F5
F6
F7
F8
F9
F10
F11
F12
F13
F14
F15
F16
F17
F18
F19
F20
F21
F22
F23
F24
F25
F26
F27
F28
F29
F30

3.97e+04±2.14e+04
0.00e+00±0.00e+00
0.00e+00±0.00e+00
1.55e+01±3.60e+01
2.02e+01±2.26e-02
2.52e+01±1.70e+00
3.48e-03±6.36e-03
0.00e+00±0.00e+00
9.63e+01±8.72e+00
7.59e-03±1.09e-02
3.76e+03±2.96e+02
1.77e-01±2.16e-02
3.87e-01±3.44e-02
2.41e-01±3.31e-02
6.92e+00±8.56e-01
1.82e+01±4.73e-01
2.87e+03±8.34e+02
1.25e+02±5.26e+01
1.15e+01±9.13e-01
9.08e+01±3.34e+01
1.23e+03±3.35e+02
4.65e+02±1.14e+02
3.44e+02±0.00e+00
2.59e+02±3.55e+00
2.06e+02±1.00e+00
1.00e+02±4.21e-02
7.94e+02±2.51e+02
1.18e+03±2.72e+01
9.08e+02±7.61e+01
8.59e+03±4.26e+02

6.02e+04±2.63e+04−
0.00e+00±0.00e+00≈
1.29e-06±9.12e-06−
2.52e+01±4.31e+01−
2.02e+01±1.80e-02≈
2.57e+01±1.67e+00≈
3.09e-03±5.09e-03≈
0.00e+00±0.00e+00≈
1.01e+02±9.22e+00−
7.35e-03±8.35e-03≈
3.81e+03±3.45e+02≈
1.71e-01±1.98e-02≈
3.77e-01±3.39e-02≈
2.46e-01±2.54e-02≈
6.88e+00±6.65e-01≈
1.82e+01±3.86e-01≈
2.89e+03±9.35e+02≈
1.14e+02±4.41e+01≈
1.16e+01±1.22e+00≈
1.20e+02±4.48e+01−
1.35e+03±5.27e+02≈
4.81e+02±1.55e+02≈
3.44e+02±0.00e+00≈
2.60e+02±3.51e+00≈
2.06e+02±1.07e+00≈
1.00e+02±4.69e-02≈
8.81e+02±2.22e+02≈
1.19e+03±3.77e+01≈
9.22e+02±9.16e+01≈
8.60e+03±4.15e+02≈

1.74e+04±1.00e+04+
0.00e+00±0.00e+00≈
0.00e+00±0.00e+00≈
2.28e+01±4.10e+01≈
2.01e+01±1.93e-02+
5.27e+00±2.05e+00+
1.55e-03±3.45e-03≈
0.00e+00±0.00e+00≈
3.31e+01±4.45e+00+
1.10e-02±1.21e-02≈
3.38e+03±3.36e+02+
1.60e-01±1.89e-02+
3.12e-01±5.14e-02+
2.88e-01±4.11e-02−
5.79e+00±6.53e-01+
1.75e+01±4.16e-01+
2.58e+03±7.50e+02≈
1.49e+02±3.78e+01−
9.68e+00±2.29e+00+
1.98e+02±6.83e+01−
1.27e+03±3.61e+02≈
3.77e+02±1.64e+02+
3.44e+02±0.00e+00≈
2.74e+02±2.42e+00−
2.12e+02±7.08e+00−
1.02e+02±1.40e+01≈
4.49e+02±5.15e+01+
1.14e+03±6.05e+01+
8.89e+02±5.30e+01≈
9.43e+03±7.18e+02−

0
5
25

13
6
11

vs. shuffle exponential + (better)(Wilcoxon rank-sum) − (worse)p < 0.05 ≈ (no sig.)References[1] J. J. Liang, B. Y. Qu, and P. N. Suganthan. Problem Definitions and Evaluation Criteria for the CEC 2014 Special Session and Competition on Single Objective Real-Parameter Numerical Optimization. Technical report,
Zhengzhou University and Nanyang Technological University, 2013.