# Using Optimisation Meta-Heuristics for the Roughness Estimation Problem in River Flow Analysis
## 6 Experimental Results  
We recorded final fitness values and corresponding positions in the search space from each run of the five selected meta-heuristics, thus allowing for statistical analysis of the optimisation results. Tables 1 and 2 report the results for the Paglia and Aniene rivers respectively and display the average final fitness value, the minimum and maximum final fitness values, and the standard deviation obtained from the 25 executions of each algorithm under study. Note that the algorithms are ordered by increasing average fitness and the best results are denoted in boldface. Moreover, the last line of these tables shows the fitness value achieved with the deterministic standard methodology, described in Section 3.1, for further comparison. From a quick inspection of Tables 1 and 2, one can immediately notice that all the five meta-heuristics largely outperform the standard method, which is outperformed also if only the worst run is considered. Since the fitness values provided in the tables are the estimation errors in meters, the accuracy gain obtained by the proposed automatic methodology is of more than 1.5 m with respect to the standard approach in both the case studies, even considering the worst execution of the less effective meta-heuristic. Regarding the results achieved by the five meta-heuristics, it is interesting to note that (1 + 1)-ES clearly outperforms all the other competitors. A possible reason for this is that the small budget of evaluations allowed--due to the intrinsic complexity of the fitness function--tends to favour trajectory-based algorithms, as (1 + 1)-ES is, to population-based ones, as the other four meta-heuristics may be considered. Besides, the worst error \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Algorithm**} & \multicolumn{3}{c}{**Final Fitness Values for the River Paglia**} \\ \cline{2-5}  & **Average** & **Minimum** & **Maximum** & **St.Dev.** \\ \hline (1 + 1)-ES & **\textless{}10\({}^{-5}\)** & **0** & **0.00005** & **\textless{}10\({}^{-5}\)** \\ DE & 0.00147 & 0.00004 & 0.00665 & 0.00211 \\ CMA-ES & 0.00167 & 0.00013 & 0.00355 & 0.00139 \\ BO & 0.01364 & 0.00538 & 0.02898 & 0.00807 \\ PSO & 0.03719 & 0.00199 & 0.08304 & 0.03013 \\ Std Method & 1.63000 & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Experimental results for the case study 1 river ‘Paglia’. For any meta-heuristic, the table provides the average, minimum, maximum and standard deviation of the fitness values observed at the end of all the 25 executions. For the standard method, which is deterministic, only one result is provided. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Algorithm**} & \multicolumn{3}{c}{**Final Fitness Values for the River Aniene**} \\ \cline{2-5}  & **Average** & **Minimum** & **Maximum** & **St.Dev.** \\ \hline (1 + 1)-ES & **0.00003** & **0** & **0.00021** & **0.00006** \\ DE & 0.00362 & 0.00020 & 0.01190 & 0.00333 \\ CMA-ES & 0.00431 & 0.00051 & 0.01572 & 0.00442 \\ PSO & 0.00815 & 0.00048 & 0.01770 & 0.00532 \\ BO & 0.01212 & 0.00085 & 0.02762 & 0.00914 \\ Std Method & 1.78000 & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Experimental results for the case study 2 river ‘Aniene’. For any meta-heuristic, the table provides the average, minimum, maximum and standard deviation of the fitness values observed at the end of all the 25 executions. For the standard method, which is deterministic, only one result is provided. obtained by execution of \((1+1)\)-ES is of just 0.021 cm--in the Aniene case study, see Table 2-, thus making \((1+1)\)-ES the algorithm to go for practitioners that want to consider our methodology for future works in flood analysis. Moreover, for the purposes of the meta-heuristic research field, we also analyse the effectiveness of all the five competitor meta-heuristics. With this aim, the pairwise comparisons among the results of the meta-heuristics provided in Tables 1 and 2 were statistically validated by means of the Kolmogorov-Smirnov (KS) statistical test [54].