## 5 Experiment and Results
### Results and Analysis
#### 5.4.1 Results Exceeding a Certain Threshold  
Table 9 shows information on how many times a given selection method exceeded a given threshold. In this case, tournament selection outperformed roulette selection, exceeding the threshold seven times compared to five. This comparison indicates that when optimising DQN hyperparameters in the CartPole environment, it is more important \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Selection** & **Crossover** & **Mutation** & **Duration** & **Score** \\ \hline tournament & multi-point & boundary & 08:59:25 & 979.16 \\ tournament & one-point & non-uniform & 14:31:33 & 971.04 \\ roulette & multi-point & non-uniform & 10:32:24 & 943.92 \\ roulette & arithmetic & boundary & 04:44:51 & 936.66 \\ roulette & multi-point & boundary & 09:46:44 & 809.66 \\ tournament & arithmetic & boundary & 02:21:06 & 759.44 \\ roulette & uniform & non-uniform & 01:04:58 & 630.04 \\ tournament & uniform & non-uniform & 02:08:04 & 605.64 \\ tournament & uniform & boundary & 01:18:32 & 498.56 \\ tournament & arithmetic & non-uniform & 01:48:23 & 479.88 \\ roulette & one-point & non-uniform & 02:56:49 & 439.00 \\ tournament & multi-point & non-uniform & 02:55:21 & 406.30 \\ roulette & one-point & uniform & 02:56:06 & 379.10 \\ roulette & multi-point & uniform & 02:45:14 & 372.08 \\ roulette & arithmetic & non-uniform & 02:10:25 & 368.84 \\ roulette & uniform & uniform & 01:52:57 & 358.64 \\ roulette & arithmetic & uniform & 01:24:43 & 352.22 \\ tournament & uniform & uniform & 02:16:08 & 293.22 \\ tournament & arithmetic & uniform & 01:52:39 & 280.42 \\ tournament & multi-point & uniform & 02:29:55 & 279.06 \\ roulette & uniform & boundary & 00:33:43 & 239.86 \\ roulette & one-point & boundary & 01:21:16 & 212.20 \\ tournament & one-point & uniform & 00:46:38 & 211.02 \\ tournament & one-point & boundary & 00:38:33 & 127.44 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Results of selection, crossover, and mutation method combinations. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Selection** & **Crossover** & **Mutation** & **Composite Score** \\ \hline roulette & arithmetic & boundary & 0.650 \\ tournament & arithmetic & boundary & 0.614 \\ roulette & uniform & non-uniform & 0.553 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Best combinations by duration and score. to explore the hyperparameter space, which is more possible with tournament selection. By contrast, roulette selection tends to follow the best individuals more closely. As can be seen in Table 10, all crossover methods met the specified threshold. Multi-point crossover emerged as the most effective method in this comparison. Crossover methods such as arithmetic and uniform achieved the same result at level 3 in terms of meeting the reward threshold conditions. The worst case is the single-point crossover method, which contains two factors that influence the optimisation. Referring to the exploration of hyperparameter space, the single-point crossover method exhibited the lowest degree of exploration among the selected crossover techniques. Table 11 lists the methods by which an agent can achieve a given reward threshold. It is surprising that the uniform method, which randomly selects new values from a specific range as part of the mutation, did not optimise the hyperparameter values. The reason for its absence in this list may be the lack of a sufficiently large step size in value selection. It is also possible that this is due to the specific configuration of this method in combination with selection and mutation, as well as the inherent randomness, which may have led to the loss of beneficial traits across generations. The non-uniform method turned out to be the best because it avoided local optima and therefore focused more on promising solution areas. The boundary method, on the other hand, suggests that the optimal values of the hyperparameters were located near the upper and lower limits of their value ranges. Figure 3 illustrates the evolution of the best-performing individuals in two GAs.