# Addressing the Algorithm Selection Problem through an Attention-Based Meta-Learner Approach
## 4 Experiments and Results  
This section provides details of the experiments conducted, their main results, and a brief discussion of the most relevant findings. We compared the proposed approach for learning the problem-algorithm mapping against five algorithm selectors frequently described in the recent literature related to this work. These algorithms contain the three main components (machine learning, meta-heuristics, and combinatorial optimization problems) identified by [PERSON] et al. [23]. Moreover, we used Bayesian optimization [47] across all models for tuning their corresponding hyperparameters. This approach for hyperparameter tuning has proven to be an effective strategy for this task [48; 49; 50]. The five baseline methods against which the proposed attention-based meta-learner (AM)--with a single attention block--is compared are a multi-layer perceptron (MLP) [51], a \(k\)-nearest neighbor selector (KNN) [52], a softmax regression (SR) [53], a support vector machine (SVM) [54] and a decision tree (DT) [55]. The optimization scheme used for setting the hyperparameters of all models consisted of 300 iterations of Bayesian optimization coupled with 33 steps of random exploration. Regarding the search strategy, we used a Gaussian process along with Tikhonov regularization [56] to inform the overall search strategy for optimal hyperparameters with the predicted uncertainty via the acquisition function. We can define this Gaussian process as:  
\[y\sim\mathcal{GP}(m(x),k(x_{i},x_{j})) \tag{4}\]  
for which \(m(x)\), the mean function, corresponds to the mean of the data and \(k(x_{i},x_{j})\), the kernel function, is a Matern covariance function [56] as defined by:  
\[k(x_{i},x_{j})=\frac{1}{\Gamma(\
u)2^{\
u-1}}\left(\frac{\sqrt{2\
u}}{l}d_{ij }\right)^{\
u}K_{\
u}\left(\frac{\sqrt{2\
u}}{l}d_{ij}\right) \tag{5}\]  
where \(\Gamma\) stands for the gamma function, \(K_{\
u}\) stands for a modified Bessel function, and \(d_{ij}\) represents the Euclidean distance between vectors \(x_{i}\) and \(x_{j}\)[56]. We set parameters \(\
u\) and \(l\) to \(\frac{5}{2}\) and 1.0, respectively. Furthermore, the parameters of the kernel were optimized by maximizing the log-marginal-likelihood via the limited-memory Broyden-Fletcher-Goldfarb-Shannon algorithm extension for handling bound constraints (L-BFGS-B) [57] with 15,000 maximum iterations and function evaluations, 20 line search steps per iteration and an absolute step size of \(1.00\times 10^{-8}\) for the numerical approximation of the Jacobian using forward differences. We conducted the training process and hyperparameter optimization of all selectors using 85.8% of the previously described dataset following a 10-fold cross-validation scheme. We reserved the remaining 14.2% of the dataset for reporting the final results of the evaluation of the selectors. This work's main point of interest is studying how often a selector chooses the correct algorithm for a particular instance that, on average, generates the best solution for a given VRPTW instance. For this reason, we decided to use accuracy as the metric to be optimized during the Bayesian optimization for the testing set across the ten folds. Table 1 showsthe hyperparameters and bounds used for each selector and the corresponding optimal values obtained. Additionally, in the case of the DT model, we used entropy as a criterion to measure the quality of splits via information gain [58; 59]. Moreover, we employed the Newton-CG algorithm for the optimization problem in the case of the SR selector [60; 61]. For the MLP method, we used the ReLU [62] activation function along with the adaptive moment estimation (Adam) [63] algorithm and the negative log-likelihood loss. Furthermore, in the case of the KNN model, a ball-tree (metric-tree) algorithm was used for computing the nearest neighbors [64; 65]. Finally, for the AM, we used the root mean squared propagation (RMSProp) [63] optimization algorithm along with the negative log-likelihood loss.