## 2 Related Works  
### Hyperparameter Optimization  
The most straightforward algorithm for hyperparameter optimization (HPO) is Grid Search, which involves an iterative evaluation of all possible combinations of hyperparameters. While this method guarantees finding the most optimal set of hyperparameters, it is the most resource-intensive one.  
Random Search [8] is another frequently used method, which allows for balancing speed and quality by reducing or increasing the number of iterations. It samples hyperparameter values randomly within specified ranges, often finding good results faster than Grid Search by covering a broader search space more efficiently. Random Search is often used for evaluating the effectiveness of other optimization algorithms.  
In addition to naive implementations of random search and grid search, methods based on continuous and discrete optimization algorithms aim to provide a more efficient way to achieve hyperparameter optimization. Some of the most popular such methods are Bayesian Optimization [16], evolutionary algorithms, and population-base heuristics [17; 18; 19]. These methods have shown a state-of-the-art performance on various benchmarks and are extensively utilized by engineers for hyperparameter optimization in convolutional neural networks.  
Also, in recent years, special modifications have been developed for hyperparameter optimization algorithms that aim to reduce the time and computational cost by using heuristics such as early stopping of training and the evaluation of unpromising configurations and parallel evaluation of multiple model configurations. Two of the most popular methods among these are hyperband [20] and sequential model-based optimization (SMBO) [21; 22].  
Active development in the field of hyperparameter optimization has led to the emergence of entire scientific and engineering directions aimed at streamlining and enhancing the pipelines of the development and optimization of neural network models. Examples of such directions are Neural Architecture Search (NAS) [23] and Automatic Machine Learning (AutoML) [24].  
TetraOpt contributes to the field of hyperparameter optimization by providing an efficient and structured algorithm based on tensor train programming. Unlike Grid Search, which exhaustively explores all configurations, TetraOpt focuses on the most promising areas of the hyperparameter space. In contrast to Random Search, TetraOpt uses a deterministic strategy based on tensor factorization to identify configurations for evaluation. The algorithm factorizes the search space into a tensor, where each element corresponds to a model configuration and its associated optimization metric (e.g., test accuracy). Using the MaxVol operation [2], TetraOpt identifies the most promising regions in this tensor and iteratively explores the configuration search space.  
One important feature of TetraOpt is its ability to produce continuous improvement as the number of iterations (sweeps) increases, which shows the algorithm's ability to avoid falling into local optima.  
Another important feature of the algorithm is that it determines the most promising areas of the hyperparameter search space using the MaxVol routine. This allows it to avoid evaluating unpromising configurations and to save computational and time resources.  
We believe this makes TetraOpt a practical method for hyperparameter optimization in neural networks.