# Using Optimisation Meta-Heuristics for the Roughness Estimation Problem in River Flow Analysis
## 6 Experimental Results
Hence, by considering a significance level of 0.01, the results of the KS tests carried out are depicted in Figure 7,b for, respectively, the river Paglia and the river Aniene. Each figure is a heatmap where a red square is used to indicate that the corresponding column-algorithm significantly outperforms the corresponding row-algorithm; a blue square means that the row-algorithm is significantly better than the column-algorithm; a grey square is used if there is no statistical difference between the two algorithms. Figure 7,b clearly confirm that \((1+1)\)-ES significantly outperforms all the other competitors. Regarding the other comparisons, a distinction must be made between the two case studies. For the river Aniene--see Figure 7--DE, CMA-ES, PSO and BO do not show significant differences in terms of effectiveness, while there are few differences in the case study Paglia--see Figure 7. This is possibly due to the larger dimensionality of the second case study (\(d=120\) vs. \(d=60\)) which reduces the number of iterations performed by the four population-based algorithms--in the allotted budget of computational time-, thus preventing them to show significant differences in the evolved population. However, in the case study Paglia--see Figure 7--, DE and CMA-ES do not show significant differences between them, but both outperform PSO and BO. The empirical probability density functions of the final fitness values obtained by the 25 executions of any single algorithm are depicted in the violin-plots of Figure 8,b for, respectively, the river Paglia and the river Aniene. Figure 7: Results of the Kolmogorov-Smirnov tests with significance level 0.01. A red entry means that column-algorithm is significantly better than row-algorithm; a blue entry means that row-algorithm is significantly better than column-algorithm; a grey entry means that there is no statistical difference between the algorithms. (**a**) Case study 1—River Paglia; (**b**) Case study 2—River Aniene. These plots clearly confirm the main observation: \((1+1)\)-ES is by a large amount the algorithm to go. Other observations are as follows:  
* DE and CMA-ES look, in general, to be preferable to PSO and BO;
* BO is not competitive enough in both case studies and, in particular, its effectiveness deteriorates in the larger case study, probably because BO is not good enough for large dimensionalities (indeed, it is generally used to tune the few hyper-parameters of machine learning models [30]);
* regarding robustness, \((1+1)\)-ES is clearly the most robust algorithm;
* the high variance for the results of the PSO algorithm may indicate the presence of a considerable amount of local optima in the fitness landscape, given that PSO is prone to prematurely converge to local optima [56];  
Finally, Figure 9a,b depict the convergence behaviours of the five algorithms averaged over the 25 executions for, respectively, the river Paglia and the river Aniene. It is interesting to observe that \((1+1)\)-ES outperforms its population-based competitors (with population size equal to 30) after only 30 fitness functional evaluations, thus showing to be the most efficient and suitable for this problem. The convergence graphs also show that, with a larger budget, it is likely that both DE and CMA-ES will be able to match the same effectiveness of \((1+1)\)-ES. However, since \((1+1)\)-ES reaches a practically null error, there is no real need to increase the computational budget. We can conclude this discussion by noting that the analyses provided take into account two different river traits and consider different amounts of cross-sections. In particular, this last characteristic influences the search space size of our methodology. Anyway, in both the case of studies--of 60 and 120 dimensions-, the \((1+1)\)-ES algorithm allows a quicker convergence and larger robustness with respect to the other meta-heuristics.