# Sampling Effects on Algorithm Selection for Continuous Black-Box Optimization
## 6 Results
### Sampling Effects on Selection

Table 4: Average \(SR\) across all for a given problem at a given dimension during HOIO and HOPO validations, and average \(BCSR\) for a given problem at a given dimension during HOPO validations. In boldface are the \(SR\) and \(BCSR\) less than 90%, and the \(WCSR\) higher than 10%. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline  & \(\overline{B}\) & \(\overline{T}_{\bar{B}}\) & \(SR_{\bar{B}}\) & \(BCSR_{\bar{B}}\) & \(WCSR_{\bar{B}}\) & \(TFR_{\bar{B}}\) \\ \hline \multirow{8}{*}{ICARUS} & & 100 & \(\langle 3,283,3.302|_{0.008}\) & \([98.1\%,99.2\%]_{0.99\%}\) & \([94.2\%,96.8\%]_{0.95}\) & \([0.8\%,2.2\%]_{1.05}\) & \([0.2\%,1.2\%]_{0.37}\) \\  & & 178 & \(\langle 3,350,3.302|_{0.019}\) & \([99.5\%,99.8\%]_{0.17\%}\) & \([94.5\%,97.5\%]_{0.95}\) & \([0.6\%,1.8\%]_{0.75}\) & \([0.3\%,1.5\%]_{0.75}\) \\  & HOIO & 316 & \(\langle 3,417,342|_{0.003}\) & \([99.6\%,100|_{0.017}\) & \([97.7\%,99.7\%]_{0.75}\) & \([0.2\%,0.6\%]_{0.75}\) & \([0.1\%,0.4\%]_{0.75}\) \\  & & 562 & \(\langle 3,509,3.51|_{0.002}\) & \([99.7\%,99.9\%]_{0.00\%}\) & \([98.4\%,99.0\%]_{0.45}\) & \([0.2\%,0.4\%]_{0.75}\) & \([0.0\%,0.2\%]_{0.75}\) \\  & & 1000 & \(\langle 3,**621,3623|_{0.000}\) & \([99.6\%,99.8\%]_{0.00\%}\) & \([98.8\%,99.1\%]_{0.75}\) & \([0.3\%,0.4\%]_{0.05}\) & \([0.1\%,0.1\%]_{0.75}\) \\ \cline{2-11}  & & 100 & \(\langle 3,263,3278|_{0.000}\) & \([93.5\%,94.5\%]_{0.33\%}\) & \([99.2\%,61.2\%]_{0.45}\) & \([0.1\%,2.2\%]_{0.33}\) & \([0.2\%,3.3\%]_{0.15}\) & \([0.7\%,1.5\%]_{0.75}\) \\  & & 178 & \(\langle 3,260,3.28|_{0.202}\) & \([91.7\%,99.6\%]_{0.75}\) & \([0.6\%,3.8\%]_{0.75}\) & \([0.2\%,5.3\%]_{0.75}\) & \([0.2\%,3.0\%]_{0.75}\) \\  & HOPO & 316 & \(\langle 3,352,3.36|_{0.012}\) & \([93.4\%,94.2\%]_{0.00\%}\) & \([60.4\%,61.7\%]_{0.75}\) & \([0.3\%,1.7\%]_{0.75}\) & \([0.3\%,4.0\%]_{0.75}\) \\  & & 562 & \(\langle 3,496,31|_{0.006}\) & \([95.3\%,96.0\%]_{0.33\%}\) & \([0.6\%,92.6\%]_{0.75}\) & \([0.4\%,5.4\%]_{0.75}\) & \([0.3\%,3.4\%]_{0.75}\) \\  & & 1000 & \(\langle 3,**611,367**|_{0.004}\) & \([92.5\%,93.0\%]_{0.00\%}\) & \([56.7\%,97.0\%]_{0.75}\) & \([0.2\%,7.7\%]_{0.75}\) & \([0.4\%,5.4\%]_{0.75}\) & \([0.4\%,5.4\%]_{0.75}\) \\ \hline \multirow{8}{*}{BELL} & & 100 & \(\langle 3,256,3.27|_{0.002}\) & \([97.7\%,98.8\%]_{0.10\%}\) & \([94.0\%,96.4\%]_{0.75}\) & \([0.7\%,1.5\%]_{0.75}\) & \([0.2\%,0.8\%]_{0.75}\) \\  & & 178 & \(\langle 3,332,353|_{0.002}\) & \([99.4\%,99.7\%]_{0.75}\) & \([94.2\%,97.3\%]_{0.75}\) & \([0.4\%,1.2\%]_{0.75}\) & \([0.2\%,0.9\%]_{0.75}\) \\  & HOIO & 316 & \(\langle 3,401,340|_{0.003}\) & \([99.5\%,99.8\%]_{0.20\%}\) & \([97.8\%,99.7\%]_{0.75}\) & \([0.3\%,0.6\%]_{0.75}\) & \([0.1\%,0.2\%]_{0.75}\) \\  & & 562 & \(\langle 3,495,3.49|_{0.002}\) & \([99.6\%,99.8\%]_{0.11\%}\) & \([98.3\%,99.0\%]_{0.45}\) & \([0.3\%,0.5\%]_{0.75}\) & \([0.1\%,0.2\%]_{0.75}\) \\  & & 1000 & \(\langle 3,**610,3612|_{0.000}\) & \([99.6\%,99.8\%]_{0.00\%}\) & \([98.8\%,99.1\%]_{0.75}\) & \([0.3\%,0.8\%]_{0.75}\) & \([0.2\%,0.2\%]_{0.75}\) \\ \cline{2-11}  & & 100 & \(\langle 3,321,325|_{0.015}\) & \([98.7\%,98.2\%]_{0.75}\) & \([0.8\%,86.0\%]_{0.75}\) & \([0.8\%,1.0\%]_{0.75}\) & \([0.8\%,1.0\%]_{0.75}\) \\  & & 178 & \(\langle 3,312,333|_{0.012}\) & \([90.1\%,91.9\%]_{0.75}\) & \([0.6\%,60.6\%]_{0.75}\) & \([0.8\%,9.4\%]_{0.75}\) & \([0.3\%,4.0\%]_{0.75}\) \\  & & 316 & \(\langle 3,357,3.37|_{0.011}\) & \([99.0\%,89.8\%]_{0.75}\) & \([59.6\%,60.7\%]_{0.75}\) & \([0.3\%,8.8\%,9.7\%]_{0.75}\) & \([0.3\%,4.3\%]_{0.75}\) \\  & & 562 & \(\langle 3,439,3.447|_{0.008}\) & \([88.8\%,89.5\%]_{0.75}\) & \([58.6\%,59.4\%]_{0.75}\) & \([0.7\%,6.\%,8.1\%]_{0.75}\) & \([0.2\%,2.9\%]_{0.75}\) \\  & & 1000 & \(\langle 3,**572,3578|_{0.003}\) & \([90.9\%,91.4\%]_{0.75}\) & \([61.6\%,62.7\%]_{0.75}\) & \([9.7\%,10.1\%]_{0.75}\) & \([5.4\%,5.8\%]_{0.75}\) \\ \hline \end{tabular}
\end{table} 
Table 5. Bootstrapped 95% CI of \(\{\overline{T},SR,RCSR,WCSR,TFR\}\). The sub-index represents the bias of the estimation. In boldface are those intervals whose range is the smallest for a given validation, and the lowest bias.