## 5 Conclusions  
### Hyperparameter Optimization  
In this paper, we evaluated the TetraOpt algorithm for hyperparameter optimization in convolutional neural network models against multiple widely used optimization methods, including Particle Swarm Optimization (PSO), Genetic Algorithm (GA), Differential Evolution (DE), Bayesian Optimization (BO), and Random Search (RS).  
Our experiments, conducted on six subsets of the NATS-Bench dataset, demonstrated that TetraOpt consistently finds hyperparameter configurations with higher accuracies compared to other algorithms across most experiments.  
It is important to notice that, unlike some of the comparison algorithms, in most experiments, TetraOpt's results did not plateau. This shows the stable ability of the algorithm to not get stuck in local optima.  
Experiments measuring total algorithm runtime also showed the competitiveness of the TetraOpt algorithm compared to other state-of-the-art optimization methods; across six experiments, TetraOpt showed its ability to find high-performing configurations with lower or comparable algorithm runtime compared to competing algorithms. This characteristic of TetraOpt is particularly useful in real-life scenarios, where time and computational resources are limited.  
Overall, these results show that TetraOpt can be utilized as practical and competitive algorithm for hyperparameter optimization alongside other widely used algorithms.  
We see the following directions when it comes to research and development of the TetraOpt algorithm in applications to the issue of hyperparameter optimization for convolutional neural networks:  
Combining TetraOpt with local search heuristics can improve algorithm performance and reduce overall runtime. Some competing algorithms, like Bayesian Optimization, showed an ability to quickly find configurations near local optima. While these algorithms reach a plateau, we can utilize these found configurations as a starting point for the TetraOpt algorithm to further explore the region and improve the results by finding results closer to the global optima. This would allow for improving TetraOpt's ability to find global optima more rapidly and effectively.  
Combining TetraOpt with optimization algorithms like Hyperband can further reduce the overall algorithm runtime. These algorithms are designed specifically for hyperparameter optimization use cases and aim to reduce the computational and time expenses of optimization algorithms by exploring multiple configurations in parallel (for instance, training and evaluating multiple models in parallel) and utilizing early stopping of the evaluation of unpromising configurations (e.g., stopping the training of models that show the lowest results concerning evaluation after an intermediate step). These features are highly compatible with the TetraOpt algorithm and their integration would allow for further enhancing TetraOpt's scalability and applicability in complex optimization tasks.  
Also, a very important direction is to investigate the performance of the TetraOpt algorithm for other metrics, such as recall or F1-score. Another interesting direction is to study TetraOpt's ability to provide the multi-objective optimization of different metrics, including metrics in the optimization process besides quality metrics, as well as metrics like algorithm cost and runtime.