## 5 Experiment and Results
### Results and Analysis
#### 5.4.1 Results Exceeding a Certain Threshold
In particular, learning rate \(=\) 0.01 achieved the highest mean score \(\approx\) 475, whereas an overly high learning rate (e.g., 0.1) or a moderately high learning rate (0.05) resulted in larger variance and generally lower mean returns. Lower learning rates such as 0.001 produced more stable but overall lower rewards (mean \(\approx\) 268). To determine whether these observed differences were statistically significant, a one-way Analysis of Variance (ANOVA) was performed across the five learning rate groups. The resulting \(p\)-value was approximately \(9.17\times 10^{-5}\), indicating that at least one pair of learning rate values yields a significantly different mean performance. A [PERSON]'s HSD test served as a post-hoc procedure to identify which specific pairs of learning rates differed. As shown in Table 14, learningrate \(=\) 0.01 notably outperforms learning rate \(=\) 0.001, 0.05, and 0.1 (\(p<0.05\)), whereas its difference from learning rate \(=\) 0.005 is not statistically significant (\(p=0.21\)). These results suggest that learning rates near 0.005-0.01 provide relatively stable and high returns, whereas much lower (0.001) or higher (0.1) values can degrade performance. A [PERSON]'s \(t\)-test was additionally performed between the extreme learning rates (0.001 and 0.1), yielding a \(p\)-value \(\approx\) 0.55, indicating no statistically reliable difference between the lowest and highest learning rate values. Notably, both of these learning \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Learning Rate** & **Count** & **Mean** & **Std. Dev.** & **Min** & **25\%** & **50\%** & **75\%** & **Max** \\ \hline
0.001 & 25 & 268.008 & 45.259 & 205.16 & 244.78 & 254.08 & 309.40 & 326.62 \\
0.005 & 25 & 361.092 & 98.712 & 253.32 & 291.04 & 322.98 & 416.70 & 521.42 \\
0.01 & 25 & 475.108 & 205.455 & 137.96 & 390.74 & 502.70 & 630.36 & 713.78 \\
0.05 & 25 & 293.292 & 180.220 & 9.54 & 235.48 & 296.32 & 376.74 & 548.38 \\
0.1 & 25 & 231.512 & 301.287 & 9.62 & 9.96 & 12.04 & 376.56 & 749.38 \\ \hline \hline \end{tabular}
\end{table} 
Table 13: Descriptive statistics of mean scores for each learning rate, aggregated over 25 runs (five seeds \(\times\) five repeats). \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Group 1** & **Group 2** & **Mean Diff** & _p-adj_ & **Lower** & **Upper** & **Reject** \\ \hline
0.001 & 0.01 & 207.10 & 0.0015 & 59.60 & 354.60 & True \\
0.01 & 0.05 & \(-\)181.82 & 0.0077 & \(-\)329.32 & \(-\)34.32 & True \\
0.01 & 0.1 & \(-\)243.60 & 0.0001 & \(-\)391.10 & \(-\)96.10 & True \\ \hline \hline \end{tabular}
\end{table} 
Table 14: Excerpt of Tukey HSD results for pairs of learning rate. Only relevant comparisons shown. rates exhibited broad variance and produced mean scores substantially below that of learning rate \(=0.01\), though in different ways: 0.001 yielded consistently moderate returns, whereas 0.1 occasionally spiked but remained unstable on average. Figure 9 shows a graph representing the cumulative rewards obtained by the agent during the training process, which lasted 150 episodes. In this case, the values of the selected hyperparameters were optimised by configuring roulette selection, arithmetic crossover, and boundary mutation. The increase in rewards starts from the 32 nd episode. The value of the reward in subsequent episodes fluctuates. Around the 95 th episode, the agent reaches the maximum score and then maintains it with decreases in some episodes. Despite the instability, the agent achieves the highest scores in the subsequent episodes, indicating that it understood the environment.