# Learning Individualized Hyperparameter Settings
## 3 Algorithms and Problems
### Robust Tabu Search and the QAP
Note that the results reported in the \(S_{0}\) column correspond to the results obtained by irace alone; therefore, the table shows a comparison of the results obtained by the individualized settings against those obtained with a state-of-the-art automatic configurator. We did not report the CPU times needed for obtaining the individualized settings because they corresponded to the time needed to propagate the input in a small three-level MLP, therefore less than 1 ms. In the case of the GAP, the in-sample validation produced the results shown in Table 1. The individualized setting produced more dominating results on all instance subsets, although statistical significance (\(\alpha=0.05\)) was never reached, albeit close for the Yagiura E subset, possibly because it counted a larger number of instances. In the case of the QAP, the in-sample validation reported in Table 2 was, as expected, less conclusive, as can be seen from the higher values of the binomial probabilities. Although the individualized setting produced dominating results in almost all rows, statistical significance was again never reached. This is not surprising, since the robust tabu search was explicitly designed to be insensitive to the parameter setting, so tuning it should not have much effect. Another possible reason for the failure to reach significance is the low numerosity of the instance sets. Given these obstacles, the consistently higher number of better results obtained with individualized settings is noteworthy. Note that the Lagrangian heuristic is deterministic, while the robust tabu search has a random component, so the results in Table 2 are six-run averages. However, it is worth noting that the robust tabu search proved to be very stable in its results, in most cases producing identical results in all repetitions. ### Out-of-Sample Validation

In out-of-sample tests, the optimized problemwide setting and the neural-suggested setting were applied to instances not used by irace in the optimization phase. This was done in two steps, first using the full datasets from the literature and then augmenting them with newly generated benchmarks. #### 5.2.1 Full Datasets

In the first step, a test set was used for the GAP that included all literature instances of the chosen subsets, not just a sample of them, although the simplest sets were quickly solved to optimality in all cases. Each instance was solved with both the global setting and the individualized setting. The results are given in Table 3 and are consistent with those of Table 1. We can see that the significance was slightly improved, mainly due to the increased cardinalities, but the relative performance was basically the same. The comparatively higher number of instances better solved by the global setting was mainly due to the fact that the tie-breaking policy acted on instances that were quickly solved to optimality. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Type** & \(n\) & \(S_{0}\) & \(S_{n}\) & \(p\) (_Binomial_) \\ \hline Beasley 11 & 5 & 1 & 4 & 0.188 \\ Beasley 12 & 5 & 2 & 3 & 0.500 \\ Yagiura D & 6 & 2 & 4 & 0.344 \\ Yagiura E & 13 & 4 & 11 & 0.059 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: GAP, in-sample validation. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Type** & \(n\) & \(S_{0}\) & \(S_{n}\) & \(p\) (_Binomial_) \\ \hline BUR & 8 & 3 & 5 & 0.363 \\ CHR & 14 & 4 & 10 & 0.090 \\ ESC & 18 & 8 & 10 & 0.407 \\ HAD & 5 & 3 & 2 & 0.813 \\ LIPA & 6 & 2 & 4 & 0.344 \\ NUG & 13 & 4 & 9 & 0.133 \\ TAI & 17 & 5 & 12 & 0.072 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: QAP, in-sample validation. Similarly, we constructed a test set including all QAPLIB instances with \(n\) between 12 and 50. This resulted in the inclusion of the sets ELS (one instance), KRA (three instances), ROU (three instances), SCR (three instances), SKO (two instances), STE (three instances), THO (two instances), and WIL (one instance).