## 6 Analysis
### Comparison of Results from Each Phase
To determine which of the results were significantly different, the post hoc test was used, the results of which are shown in Table 12. The table shows that the results in the row are inferior to those in the column when indicated by \"\(<\)\", superior when indicated by \"\(>\)\", or equivalent when indicated by \"\(\approx\)\". We can see that the results obtained using only the atomic nodes, without optimising the terminal set, usually led to the worst results. In contrast, there was no significant difference between the results from Phases 1, 2, and 3. \begin{table}
\begin{tabular}{l l l} \hline \hline  & **Median** & **Rule Size** \\ \hline EDD & 14.53 & 1 \\ COVERT & 14.20 & 19 \\ ATC & 13.73 & 16 \\ \hline Phase 0 & 14.08 & 43.2 \\ Phase 1 & 13.69 & 42.7 \\ Phase 2 & 13.65 & 43.4 \\ Phase 3 & 13.47 & 43.7 \\ Phase 2 + 3 & 14.22 & 43.8 \\ \hline \hline \end{tabular}
\end{table} 
Table 11: Comparison of results from all phases on the validation set. Figure 3: Box plot of the best results obtained by each terminal set on the validation set. The above results summarise the validation set. In order to more objectively evaluate the performance of GP with the different terminal sets, the rules obtained were evaluated using the test set. These results are listed in Table 13. It can be seen from the table that the performance of the rules obtained is now quite different from before. In this case, the results before feature selection only outperformed the simplest DR but not the two more complicated ones, namely COVERT and ATC. Even after Phases 1 and 2, GP could only find rules that performed similarly well to the manually designed rules. Only when the manually designed terminal nodes were finally introduced did GP generate rules that clearly outperformed the manually designed rules. The results are also shown as a box plot in Figure 4 to better illustrate the differences between the results. The figure illustrates that the introduction of manually designed composite terminal nodes led to the greatest increase in performance. To confirm that the addition of manually designed composite nodes led to a significant improvement in the results, the Kruskal-Wallis test was used, and a \(p\)-value of 0 was obtained, meaning that there was a significant difference. The results of the post hoc tests performed are shown in Table 14. The table shows that although in most cases there was no significant difference between the terminal sets, the use of manually designed composite terminal nodes significantly improved the results compared to those obtained in Phases 0, 1, and 2. \begin{table}
\begin{tabular}{l l l} \hline \hline  & **Median** & **Rule Size** \\ \hline EDD & 17.31 & 1 \\ COVERT & 16.86 & 19 \\ ATC & 16.63 & 16 \\ \hline Phase 0 & 17.11 & 43.2 \\ Phase 1 & 16.93 & 42.7 \\ Phase 2 & 16.71 & 43.4 \\ Phase 3 & 15.89 & 43.7 \\ Phase 2 + 3 & 16.45 & 43.8 \\ \hline \hline \end{tabular}
\end{table} 
Table 13: Comparison of results from all phases on the test set. Figure 4: Box plot of the best results obtained by each terminal set on the test set. Based on the previous results, we can see that the manually designed terminal nodes actually provided the most valuable information about the problem, allowing the rules to generalise well to unseen problem instances. Presumably, all methods performed similarly on the validation set, as the best terminal sets were selected for them. However, it appears that this may have led to a slight overfitting to the validation set. Based on the previous observations, it is, therefore, safe to conclude that a mixture of simple and manually designed composite terminal nodes is required to create the best DRs.