## 5 Results and Discussion
### CNN on the MNIST Dataset  
This section summarizes the performance accuracy obtained when each of the optimization technique (SDSGA, GA, PSO, BBO and BO) were used to tune the HPs of the CNN on MNIST datasets. Due to the use of an early stopping strategy during the model training, if no further improvement is achieved, the execution time becomes a poor metric of evaluation. As an example, we discovered that, if the validation accuracy has not increased to 70% after three epochs of training, it fails to reach 90% even after 10 to 20 epochs. As a result, continuing to use such HP combinations wastes computational resources. Therefore, a better approach to measure the performance entailed selecting a baseline validation accuracy and comparing how many iterations it takes the algorithms to give such results. Table 3 shows the results of optimizing the CNN model on the MNIST dataset. It was observed that the BO algorithm performed the least with a value of 99.14%, which is slightly less than the PSO algorithm at 99.15% while the SDSGA performed the best with an average value of about 99.2% (see Table 3). Similarly, the BBO algorithm recorded the next best in terms of performance with about 99.18%, while the GA performed at 99.17%. Furthermore, it was discovered that the differences between the optimization algorithms are not significant. This could be due to the number of fitness evaluations chosen as 150 or it could be due to the optimal accuracy of the ML model having plateaued around that value. Nonetheless, we discovered that increasing the number of fitness evaluations beyond 150 resulted in no further improvement in the results.  
The BO algorithm performed the worst in terms of the number of fitness evaluations required for the algorithms to reach optimal, with 99.14% at the tenth iteration. This means that the BO had performed 150 fitness evaluations to reach this value. The best performing algorithm was the SDSGA, which achieved an accuracy of 99.2% at the fifth iteration. The BBO algorithm performed the next best and achieved an accuracy of 99.18% at the ninth iteration, whereas the GA achieved 99.17% at the second iteration. The accuracy of the BBO algorithm shows that it performed better than the GA and PSO algorithm. Despite gradually improving, its results have been consistently optimal across all runs. Figure 6 depicts a graph of the optimization algorithms' accuracy as iteration progresses.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **PSO (\%)** & **GA (\%)** & **BBO (\%)** & **BO (\%)** & **SDSGA (\%)** \\ \hline
1 & 99.04 & 99.04 & 99 & 98.95 & 99.04 \\
2 & 99.04 & 99.17 & 99.12 & 98.99 & 99.18 \\
3 & 99.04 & 99.17 & 99.12 & 98.99 & 99.18 \\
4 & 99.04 & 99.17 & 99.12 & 98.99 & 99.18 \\
5 & 99.14 & 99.17 & 99.16 & 99.01 & 99.2 \\
6 & 99.14 & 99.17 & 99.16 & 99.05 & 99.2 \\
7 & 99.14 & 99.17 & 99.16 & 99.05 & 99.2 \\
8 & 99.14 & 99.17 & 99.17 & 99.06 & 99.2 \\
9 & 99.15 & 99.17 & 99.18 & 99.06 & 99.2 \\
10 & 99.15 & 99.17 & 99.18 & 99.14 & 99.2 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Result of optimizing the CNN model on the MNIST dataset. The data have been prepared to show the best accuracy so far.