# A Feature-Independent Hyper-Heuristic Approach for Solving the Knapsack Problem
## 4 Experiments and Results
### Confirmatory Experiments
#### 4.3.2 Confirmatory Experiment II  
In the previous experiment, we evaluated the performance of HHs trained on balanced sets of instances when tested on hard instances, and we observed a limited capacity to deal with such instances. In this experiment, we show that this behaviour is not due to the hardness of the instances, but because of the discrepancy between the instances used for training and the ones used for testing. For this reason, this experiment is twofold: (1) we test how HHs behave when trained and tested on hard instances (set \(\beta_{1}\)) and (2) we try to reduce the effect of MaxPW in the resulting HHs by reducing the number of instances in the training set. Then, we produced 30 new HHs using only 60% of the instances in set \(\beta_{1}\) for training purposes. As in previous experiments, the 60% instances were randomly selected once and used for producing the 30 HHs. For consistency, the maximum number of iterations for the EA was set to 200 and all the HHs were initialized to a cardinality of 12. The HHs produced in this experiment were evaluated on the remaining 40% of the instances in set \(\beta_{1}\). The results for rankings and total profit derived from this experiment are summarized in Table 8.  
This experiment confirms the importance of the instances used for training and their similarity with those solved during the test process. This time, the performance of most of the hyper-heuristics produced using 60% of set \(\beta_{1}\) when tested on the remaining 40% of set \(\beta_{1}\) improve on MaxPW (the best heuristic for this set). In fact, the performance of Worst-HH is practically the same as MaxPW (Figure 5). Although the results suggest that an improvement in the total profit can be obtained by using the proposed hyper-heuristic approach, the gain in profit derived from using Best-HH instead of MaxPW is rather small (\(<\)0.5%). However, this result should not be interpreted as an indication that Best-HH is replicating the behavior of MaxPW. In fact, their overall profit is similar (with a small gain for Best-HH but their specific performance is not. As depicted in Figure 6, when the ranks are analyzed, we observe that Best-HH obtains better results in more cases than the best individual heuristic, MaxPW.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & \multicolumn{4}{c}{**Heuristics**} & \multicolumn{4}{c}{**Hyper-Heuristics**} \\ \cline{2-7}
**Rank** & Def & MaxP & MaxPW & MinW & Best-HH & Median-HH & Worst-HH \\ \hline
1.0 & 16 & 54 & 3 & 3 & 12 & 4 & 2 \\
1.5 & 1 & 0 & 6 & 5 & 13 & 3 & 16 \\
2.0 & 14 & 8 & 10 & 9 & 29 & 28 & 20 \\
2.5 & 2 & 5 & 47 & 3 & 51 & 59 & 59 \\
3.0 & 14 & 16 & 54 & 30 & 67 & 64 & 56 \\
3.5 & 3 & 3 & 20 & 0 & 19 & 25 & 22 \\
4.0 & 19 & 13 & 24 & 12 & 16 & 21 & 13 \\
4.5 & 4 & 1 & 29 & 11 & 18 & 20 & 21 \\
5.0 & 30 & 19 & 10 & 19 & 10 & 8 & 11 \\
5.5 & 9 & 3 & 21 & 13 & 4 & 6 & 6 \\
6.0 & 101 & 16 & 9 & 28 & 1 & 0 & 10 \\
6.5 & 6 & 4 & 6 & 10 & 0 & 2 & 2 \\
7.0 & 21 & 98 & 1 & 97 & 0 & 0 & 2 \\ \hline Profit & 1,584,007 & 1,540,100 & 1,673,387 & 1,600,015 & **1,681,403** & 1,681,127 & 1,678,375 \\ \hline \hline \end{tabular}
\end{table}
Table 8: Ranks and total profit obtained by the four heuristics and the best, median, and worst hyper-heuristics trained on 60% of set \(\beta_{1}\) and tested on the remaining 40% of set \(\beta_{1}\). The best result, measured in terms of total profit, is highlighted in bold.