# Sampling Effects on Algorithm Selection for Continuous Black-Box Optimization
## 3 Methods
### Exploratory Landscape Analysis Features  
As inputs to the selector, we use the features summarized in Table 1, as they are quick and simple to calculate. Furthermore, these features can share a sample obtained through a Latin Hypercube Design (LHD), guaranteeing that the differences observed between features depend only on the instance, and not on sample size or sampling method. For example, the convexity and local search methods described by [8] use independent sampling approaches; hence, they cannot share a sample between them nor with the methods in Table 1. The convexity method takes two candidates, \(\left\{\mathbf{x}_{i},\mathbf{x}_{j}\right\}\), and forms a linear combination with random weights, \(\mathbf{x}_{k}\). Then, the difference between \(y_{k}\) and the convex combination of \(y_{i}\) and \(y_{j}\) is computed. The result is the number of iterations out of 1000 in which the difference is less than a threshold. Meanwhile, the local search method uses the Nelder-Mead algorithm, starting from 50 random points. The solutions are hierarchically clustered to identify the local optima of the function. The basin of attraction of a local optimum, \(\mathbf{x}_{i}\), i.e., the subset from \(\mathcal{X}\) from which a local search converges to \(\mathbf{x}_{i}\), is approximated by the number of algorithm runs that terminate at each \(\mathbf{x}_{i}\). Both sampling approaches do not guarantee the resulting sample is unbiased; hence, reusable. Besides ensuring a fair comparison, sharing a LHD sample reduces the overall computational cost, as no new candidates must be taken from the space. Figure 3: Structure of the multi-class learning model for \(A=\left\{\alpha_{1},\alpha_{2},\alpha_{3},\alpha_{4}\right\}\). The six BC models take the vector ELA features c as input and generate a probability score, \(\mathcal{R}\left(\alpha_{i},\alpha_{j}\right)\), which are used to generate a recommended algorithm \(\alpha_{b}\) as output. The adjusted coefficient of determination, \(\bar{R}^{2}\), measures the fit of linear or quadratic least squares regression models. This approach estimates the level of modality and the global structure [8], and it can be thought of as measuring the distance between a problem under analysis and a reference one [29]. To provide information about variable scaling, the ratio between the minimum and the maximum absolute values of the quadratic term coefficients in the quadratic model, \(CN\), is also calculated [8]. The mutual information can be used as a measure of non-linear variable dependency [36]. Let \(\mathcal{V}=\{1,\ldots,D\}\) be a set of variables indexes, and \(V\subset\mathcal{V}\) be a combination of such variables. The information significance, \(\xi(V)\), is calculated as follows:  
\[\xi(V)=\frac{I(\mathbf{X}_{V};\mathbf{Y})}{H(\mathbf{Y})} \tag{1}\]  
where \(H(\mathbf{Y})\) is the information entropy, \(I(\mathbf{X}_{V};\mathbf{Y})=H(\mathbf{X}_{V})+H(\mathbf{Y})-H(\mathbf{X}_{V},\mathbf{Y})\) is the mutual information, with \(\mathbf{Y}\subset\mathcal{Y}\) being the cost of \(\mathbf{X}\). The information entropy of a continuous distribution is estimated through a \(kd\)-tree partition method [37] and it is not bounded between \([0,1]\). Let \(k=|V|\) be the order of information significance. The mean information significance of order \(k\) is defined as:  
\[\xi^{(k)}=\frac{1}{\binom{D}{k}}\sum_{V\subset\mathcal{V},|V|=k}\xi(V) \tag{2}\]  
Please note that as \(D\) increases the number of possible variable combinations increases. Therefore, we limit ourselves to \(k=\{1,D\}\). The probability distribution of \(\mathcal{Y}\) may indicate whether the function is neutral, smooth or rugged, as well as the existence or lack of a global structure [8]. The shape of the distribution is characterized by estimating the sample skewness, \(\gamma(\mathbf{Y})\), and kurtosis, \(\kappa(\mathbf{Y})\)[8], and entropy, \(H(\mathbf{Y})\)[38]. Other reported measures with similar sampling simplicity were discarded as they are co-linear with those in Table 1[16].