# A Feature-Independent Hyper-Heuristic Approach for Solving the Knapsack Problem
## 4 Experiments and Results
### Confirmatory Experiments
#### 4.3.1 Confirmatory Experiment I  
So far, we have evaluated the performance of HHs on instances similar to the ones used for training, under various scenarios. In the first confirmatory experiment, we use all the instances from set \(\alpha_{1}\) to train the HHs, but test their performance on set \(\beta_{1}\). Let us recall that set \(\beta_{1}\) contains 600 hard instances proposed by [PERSON] [45] and feature 20 items and different capacities. The relevant issue regarding set \(\beta_{1}\) is that such instances are considered hard to solve.  
Table 7 shows that [PERSON] is, once again, the best individual performer in this experiment. Although all the produced HHs perform similarly (cf. Worst-HH), they are unable to improve upon the results obtained by the best heuristic. Nonetheless, all the HHs produced obtained a total profit larger than those obtained by the remaining heuristics.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & \multicolumn{5}{c}{**Heuristics**} & \multicolumn{5}{c}{**Hyper-Heuristics**} \\ \cline{2-7}
**Rank** & Def & MaxP & MaxPW & MinW & Best-HH & Median-HH & Worst-HH \\ \hline
1.0 & 83 & 99 & 111 & 96 & 0 & 0 & 0 \\
1.5 & 3 & 0 & 2 & 2 & 1 & 2 & 2 \\
2.0 & 13 & 2 & 219 & 1 & 49 & 39 & 8 \\
2.5 & 0 & 0 & 5 & 3 & 19 & 32 & 31 \\
3.0 & 1 & 14 & 30 & 19 & 82 & 61 & 37 \\
3.5 & 0 & 0 & 5 & 1 & 49 & 87 & 106 \\
4.0 & 1 & 4 & 14 & 16 & 70 & 51 & 59 \\
4.5 & 0 & 0 & 1 & 3 & 33 & 67 & 84 \\
5.0 & 0 & 26 & 13 & 91 & 53 & 33 & 42 \\
5.5 & 0 & 1 & 0 & 12 & 17 & 18 & 22 \\
6.0 & 17 & 176 & 0 & 129 & 18 & 6 & 5 \\
6.5 & 0 & 0 & 0 & 6 & 7 & 2 & 3 \\
7.0 & 282 & 78 & 0 & 21 & 2 & 2 & 1 \\ \hline Profit & 241,081 & 343,872 & **467,145** & 376,148 & 397,373 & 395,586 & 391,940 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Ranks and total profit obtained by the four heuristics and the best, median, and worst hyper-heuristics trained on set \(\alpha_{1}\) and tested on set \(\alpha_{2}\), but MaxPW is not available for the hyper-heuristics. The best result, measured in terms of total profit, is highlighted in bold.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & \multicolumn{5}{c}{**Heuristics**} & \multicolumn{5}{c}{**Hyper-Heuristics**} \\ \cline{2-7}
**Rank** & Def & MaxP & MaxPW & MinW & Best-HH & Median-HH & Worst-HH \\ \hline
1.0 & 24 & 116 & 32 & 10 & 42 & 39 & 31 \\
1.5 & 16 & 20 & 71 & 11 & 60 & 39 & 25 \\
2.0 & 32 & 41 & 111 & 79 & 124 & 57 & 33 \\
2.5 & 14 & 19 & 80 & 27 & 56 & 54 & 54 \\
3.0 & 54 & 47 & 63 & 38 & 59 & 84 & 77 \\
3.5 & 18 & 4 & 62 & 16 & 48 & 47 & 43 \\
4.0 & 69 & 33 & 47 & 35 & 53 & 98 & 79 \\
4.5 & 30 & 3 & 42 & 14 & 32 & 44 & 47 \\
5.0 & 63 & 31 & 26 & 48 & 46 & 73 & 94 \\
5.5 & 22 & 4 & 31 & 17 & 21 & 13 & 24 \\
6.0 & 190 & 43 & 20 & 64 & 36 & 42 & 75 \\
6.5 & 15 & 6 & 13 & 14 & 5 & 2 & 3 \\
7.0 & 53 & 233 & 2 & 227 & 18 & 8 & 15 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Ranks and total profit obtained by the four heuristics and the best, median, and worst hyper-heuristics trained on set \(\alpha_{1}\) and tested on set \(\beta_{1}\). The best result, measured in terms of total profit, is highlighted in bold.  
Even if the HHs were incapable of overcoming MaxPW in this experiment, it is interesting to see how close their performance is, especially as these HHs were trained with instances of a different type than those used for testing.