### Hyperparameter Optimization (HPO)  
Hyperparameters are predefined values that govern the behavior of a model and remain fixed throughout the training process. These include variables such as the learning rate, decay rates, the choice of optimizer, the number of convolutional layers in a neural network, or the sizes of individual layers. Typically, these parameters are chosen by experts based on prior knowledge and through trial-and-error experimentation. While this approach helps narrow the search space, it often risks yielding a suboptimal model configuration.  
Hyperparameter optimization aims to automate the process of identifying the optimal hyperparameter set, reducing reliance on expert input while systematically exploring the configuration space. Although this method increases computational demands due to the need to evaluate numerous model configurations, it can result in significantly improved model performance by identifying better hyperparameter settings.  
In our work, we explore the potential benefits of Tensor Network-based methods for automating the search within the configuration space of machine learning models. Our findings demonstrate a reduction in computational complexity, particularly in terms of expected runtime and the number of evaluations required for various model configurations. This reduction addresses the significant computational costs that motivated our investigation.  
In our work, we investigated TetraOpt [1], an optimization algorithm based on Tensor Train programming, for the task of hyperparameter optimization in convolutional neural networks.  
The core idea of our method is to represent the hyperparameter search space into a tensor, where the elements represent specific hyperparameter configurations and their corresponding values for the target optimization metric. By exploring multiple regions of this tensor simultaneously, we identify the most promising areas for further investigation using the MaxVol operation [2]. Through this iterative process, the algorithm converges towards the global optimum.  
TetraOpt has a deterministic mathematical complexity and its iterative design allows it to enhance the results by increasing number of algorithm iterations (\"sweeps\").  
Identification of unpromising areas using the MaxVol operation allows the TetraOpt algorithm to significantly reduce computations by avoiding the evaluation of hyperparameters from these areas. This is crucial in tasks of HPO for neural networks where each evaluation requires time- and energy-consuming model training.  
For the evaluation of TetraOpt for HPO in convolutional neural networks, we used NATS-Bench [3] (Neural Architecture Transferability and Search Benchmark)--a comprehensive benchmark for HPO for image classification tasks. To provide a comparative analysis, along with TetraOpt, we evaluated the performance of other algorithms for HPO in neural networks: Bayesian Optimization [4], Particle Swarm Optimization (PSO) [5], Differential Evolution (DE) [6], the Genetic Algorithm [7], and Random Search [8].  
TetraOpt outperformed other algorithms in finding configurations with higher accuracies in most of the experimental cases. It is important to notice that TetraOpt performance results show that algorithm steadily improves results in terms of the number of algorithm steps without stagnating or plateauing.