## 3 Numerical Analysis
### Experimental Protocol
#### 3.2.2 Performance Metrics  
Two performance metrics are observed: the objective function value of the best solution so far and the population diversity. The population diversity allows us to check if all clouds of the population have converged to the same design, which may impair the ability of the search to locate the optimal solution if this convergence is premature. The diversity can be calculated at each iteration in the following way:  
\[\text{Div}(\text{pop})\ =\ \frac{1}{\mu}\sum_{X_{i}\in\text{pop}}W_{2}^{2}(P_{ X^{*}},P_{X_{i}})\, \tag{4}\]  
where \(\text{pop}=\{X_{i},i=1,...,\mu\}\) is a population of sets, \(P_{X_{i}}\) the associated discrete measures, and \(P_{X^{*}}\) is the Wasserstein barycenter of the clouds of pop. The support's size of \(P_{X^{*}}\) is chosen to be the mode of all sizes in pop. As an alternative to the population diversity, one can look at the variance of the objective function values for the clouds in the population. Although it is cheaper to calculate than the diversity, it is more difficult to interpret as its scale depends on the objective function, if the output functions are not normalized. A very flat function would have a low population function variance even if its population is diverse. The population diversity and the best so far objective function provide complementary information. On the one hand, when an evolutionary algorithm has a high performance and a low diversity, it is likely converging towards a global optimum. On the other hand, a low population diversity associated with a poor best objective is a sign that convergence is occurring prematurely to a non-optimal set. Other diversity-performance configurations can occur: for example, a very explorative algorithm maintains by definition a high population diversity, yet it can be high performing if its search operators are biased towards the global optima.  
\begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Experiments** & **Compared Algorithms** & **The Scientific Questions** \\ \hline Alternating vs. successive Boundary and Full Domain mutations & WBGEA\_1, WBGEA\_2, WBGEA\_3 & Are there major differences between the three mutations in WBGEA? \\ \hline Handling of set size in mutation: ternary vs. full size & WBGEA\_1, WBGEA\_1t & How does the size of \(X_{rand}\) affect the performances of the algorithms? \\ \hline Wasserstein-based vs. reference evolutionary algorithm & WBGEA\_1t, Ref\_alg & How does the default Wasserstein-based evolutionary algorithm compare to a more classical evolutionary algorithm? \\ \hline Wasserstein vs. classical evolution operators & WBGEA\_1t, Ref\_alg, Ref\_rass, Wass\_gauss & How do the different operators behave when composed together differently? \\ \hline Role of crossover & WBGEA\_1t, WBGEA\_1t, rc, Ref\_alg & Are there noticeable differences between the random and the equal weight crossovers in WBGEA? Does the crossover play a major role in the performance of the algorithms? \\ \hline Role of the boundary mutation & WBGEA\_1t\_nc with four different prob: 1, 0.5, 0.1, 0.05 and 0 & What happens if we diminish the chances of performing a boundary mutation in the absence of crossover? \\ \hline \hline \end{tabular}
\end{table}
Table 2: The different experiments and the associated scientific questions.  
In the numerical experiments reported later, the mean and the standard deviation of both metrics, the best so far objective function and the population diversity are estimated over the 20 repetitions for each algorithm's variant and each test function.