# Strategies of Automated Machine Learning for Energy Sustainability in Green Artificial Intelligence
## 3 Hyperparameter Optimization Problem
### Relationship between Hyperparameter Configuration and Convergence Efficiency  
The algorithms HyperBand, BOHB, PBT, and ASHA are specifically designed to optimize hyperparameters in the field of machine learning and AutoML. Each of these algorithms implements a unique strategy to reduce the time needed for convergence.  
* HyperBand is an algorithm that utilizes the concept of non-stochastic bandits to expedite the process of random search. This is achieved through the adaptive allocation of resources and the implementation of early stopping. The problem of hyperparameter optimization is formulated as a pure exploration problem involving non-stochastic infinite-armed bandits. In this scenario, a predefined resource, such as iterations, data samples, or features, is allocated to configurations that have been randomly sampled.
* BOHB is an algorithm that amalgamates the strengths of both HyperBand and Bayesian optimization. This results in a robust and efficient method that consistently outperforms both Bayesian optimization and HyperBand across a diverse range of problem types.
* PBT, an acronym for population-based training, is an evolutionary approach that continually adjusts a set of hyperparameters during the training of a model. This is in contrast to traditional methods that maintain a fixed set of hyperparameters. PBT allows for the evolution of these hyperparameters over time.
* ASHA, short for the asynchronous successive halving algorithm, is an asynchronous variant of HyperBand. This algorithm leverages parallelism and aggressive early stopping to tackle large-scale hyperparameter optimization problems. ASHA surpasses existing methods of hyperparameter optimization, scales linearly with the number of workers in distributed environments, and is well-suited for massive parallelism.  
The reduction in convergence time can be formalized by the relationship between the number of evaluated configurations and the necessary training time to achieve a desired performance (Equation (4)). For instance, if \(T\) represents the total training time and \(N\) denotes the number of configurations, algorithms such as HyperBand and ASHA aim to minimize \(T\) while maximizing the quality of the selected configuration. This can be expressed as follows:  
\[\min_{1\leq i\leq N}T_{i}\quad\text{subject to}\quad\text{performance}(configuration_{i}) \geq\text{desired threshold} \tag{4}\]  
Here, \(T_{i}\) is the training time for configuration \(i\), and \(\text{performance}(configuration_{i})\) is a measure of quality for configuration \(i\).  
The algorithms HyperBand, BOHB, PBT, and ASHA, among others in the field of hyperparameter optimization, do not necessarily guarantee a reduction in convergence time in all cases. This is because it depends on the nature of the hyperparameter space and the objective function. However, they have demonstrated their effectiveness across a broad spectrum of problems and are particularly beneficial when parallel or distributed computational resources are available [11; 12; 58; 59].