## 4 Experimental Results
### IGD+ Analysis  
Table 5 presents IGD+ averages. From the experiments, we could conclude the following:  
* If we compare the results obtained _just by the five MOEAs_, we could see that: (i) GDE3 has better averages on five problems (P01, P04, P14, P16, and P17); (ii) IBEA excelled in four problems (P7, P11, P13, and P15); (iii) NSGAII has obtained good results in two problems (P06 and P18); (iv) SPEA2 has obtained the best IGD+ values for the five problems (P02, and P8-P11); and (v) mIBEA has obtained the best results in two problems (P03 and P05). * If we consider _just the four hyper-heuristics_, we have: (i) MOABHH excels in seven problems (P01, P07, P08, P10, P12, P14, and P17); (ii) HHLA performs better in four problems (P05, P11, P15, and P18); (iii) HHRL performs better in six problems (P02-P04, P09, P11, and P13); and (iv) HHCF has good results in two problems (P06 and P16). * Finally, if we compare the results obtained by _all nine algorithms_, GDE3 is the best algorithm on three problems (P04, and P16-P17), IBEA performs better on three problems (P07, P11, and P15), NSGAII on 2 problems (P06 and P18), SPEA2 excels on four problems (P02, P08, P09, and P11), mIBEA just on P05, MOABHH performs better on four problems (P01, P10, P12, and P14), and HHRL also in two problems (P03 and P13). Moreover, HHLA and HHCF (both among HHs) did not excel in any problem. \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Problem** & **MOABHH** & **HHLA** & **HHRL** & **HHCF** \\ \hline P01 & & & & \\ P02 & & & & \\ P03 & & & & \\ P04 & & x & & \\ P05 & & & x & \\ P06 & & x & x & \\ P07 & & x & & x \\ P08 & & x & & x \\ P09 & & x & & x \\ P10 & x & x & x & \\ P11 & x & & & x \\ P12 & & x & & \\ P13 & & & & \\ P14 & x & & & \\ P15 & & & & \\ P16 & & & & x \\ P17 & x & x & x & x \\ P18 & x & x & x & x \\ \hline
**Total** & **5** & **8** & **4** & **8** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Summary of the statistical pairwise comparison between HHs and individual MOEAs considering Hypervolume averages. An x means that the HH could not achieve a result as good as the one obtained by an MOEA. Finally, as done in Section 4.1, it is also interesting to compare the results obtained by HHs with individual MOEAs. Table 6 presents a summary of the statistical pairwise comparison between HHs and individual MOEAs considering IGD+ averages. An x in a certain cell means that the corresponding HH could not achieve a result as good as the one obtained by a particular MOEA, considering the IGD+ indicator. First of all, we can notice that all HHs were overcome in problems P15, P17, and P18. Moreover, MOABHH could not achieve statistically tied results with the best algorithms in three other problems (P03, P08, and P11), HHLA was also bet in three other problems (P04, P06, P08), HHRL could not achieve the best result in four other problems (P06, P07, P14, and P16), and HHCF did not get good results in six other problems (P05, P07, P08, P10, P11, and P13). \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Problem** & **MOABHH** & **HHLA** & **HHRL** & **HHCF** \\ \hline P01 & & & & & \\ P02 & & & & & \\ P03 & x & & & & \\ P04 & & & x & & \\ P05 & & & x & & x \\ P06 & & & x & x & \\ P07 & & & x & x & x \\ P08 & x & & x & & x \\ P09 & & & & & x \\ P10 & & & & & x \\ P11 & x & & & & x \\ P12 & & & & & x \\ P13 & & & & x & \\ P14 & & & & x & \\ P15 & x & x & x & x \\ P16 & & & & x & \\ P17 & x & x & x & x \\ P18 & x & x & x & x \\ \hline
**Total** & **6** & **6** & **7** & **9** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Summary of the statistical pairwise comparison between HHs and individual MOEAs considering IGD+ averages. An x means that the HH could not achieve a result as good as the one obtained by a MOEA.