## 2 Related Work
### State of the Art  
A wide range of hyperparameter optimization methodologies has recently developed. We only examine supporting structures for hyperparameter optimization because of space limits, and we ignore alternative methodologies for generic AutoML, including Auto-Sklearn [12] and AutoGluon [13]. Ray Tune [14] which employs Ray [15] as the back-end to spread the hyperparameter optimization process, is perhaps the most comparable to SyneTune. RayTune can run a variety of hyperparameter optimization techniques, but it cannot conduct multi-objective optimization, restricted maximization, transfer learning and evaluation. Tree Parzen Estimator approach [4], Covariance Matrix Adaptation-Evolution Strategy (CMA-ES) [16], and even multi-objective optimization are all supported by Optuna [17], but are not contemporary multi-fidelity techniques such as Asynchronous Successive Halving Algorithm (ASHA) [18] and Population Based Training (PBT) [19]. SMAC3 [20] supports multi-fidelity methods such as Bayesian Optimization using Hyperband (BOHB) [21] and Hyperband [22]; however, these are not multi-objective or restricted optimization methods. It also lacks support for asynchronous parallel methods like neural network incremental backups and asynchronous scheduling. Dragonfly [23] supports decentralized-based hyperparameter optimization techniques, but not multi-objective or transfer learning situations. HyperTune [24] has several asynchronous multi-fidelity techniques based on successive halving; however, it does not enable decentralized tweaking across several computers or provide integrated benchmarks for large-scale investigations that can be replicated. Bayesian Optimization libraries such as BOTorch, are loosely linked [25]. However, rather than decentralized hyperparameter optimization, their primary goal is to provide a framework for Monte Carlo bayesian optimization research. Hyperparameter optimization has been effectively implemented using Bayesian optimization (BO) [26]. Hyperband [22] automatically assigns resources to a collection of arbitrary combinations and employs the consecutive halving technique [27] to end poorly-performing combinations in advance, rather than using comprehensive evaluations. Bayesian optimization-based hyperband [21] enhances hyperband by using Bayesian optimization instead of random sampling. Two techniques [28; 29] suggest using learning curve extrapolation to advise early halting. A median stopping criterion is also included in Vizier [30], RayTune [14], and OpenBox [31] to halt the assessments early. Furthermore, multi-fidelity approaches [32; 33] use moderate data from partial evaluations to drive the search for the best objective function F. MFES-HB [33] blends hyperband with a Bayesian optimization based on multi-fidelity surrogates. Various approaches [34; 35] may analyze multiple combinations simultaneously rather than progressively. Furthermore, most of them [35], including BOHB [21] are focused on creating groups of configurations to analyze all at once, and just a handful offer asynchronous scheduling. Asynchronous evaluation methodology based on sequential halving method [27] is introduced by ASHA [18]. Furthermore, many asynchronous parallel computation techniques [36] cannot utilize multiple objective fidelity; A-BOHB [37] provides asynchronous multi-fidelity hyperparameter adjustment. A frequent tuning application for neural networks is searching for architectural hyperparameters. Recent empirical investigations [38] indicate that sequential Bayesian optimization methods [39; 40] outperform a variety of Neural Architecture Search (NAS) approaches [41; 42] emphasizing the need for parallelization-based bayesian optimization methods. There are two types of comprehensive evaluation-based optimization and partially based optimization methods, Bayesian optimization [43], rule-based search, genetic algorithm [44], random search [45], and other autonomous hyperparameter optimization approaches exist.