## 2 Related Work  
Several techniques have been proposed for optimizing the hyperparameters (HPs) of ML models. These include the grid search, random search, and Bayesian optimization techniques. The grid search approach was observed to be inefficient and time consuming in optimizing certain hyperparameters [14; 15]. Similarly, the random search approach has been shown to be effective in optimizing the HPs of ML models within a small search space [9]. However, due to the random nature of the technique, its performance tends to be spurious and erratic, necessitating the use of more computational resources [15]. This is frequently associated with a lack of indicators and a guide for an optimization process that governs the expected improvement direction. The Bayesian Optimization and Gaussian process approaches have also been used to optimize the HP values of ML models [16]. These algorithms are inherently sequential in nature since they rely on the results of the previous iteration to decide on the next HP sample to be evaluated [17]. Thus, this leads to their inability to use multiple CPU and GPU cores in speeding up the optimization process. Other HP optimization methods for speeding up the search space are documented in [14; 18], with a popular approach being the Successive Halving Algorithm SHA [19]. In this approach, the HPs optimization problem was defined as a non-stochastic multi armed bandit problem, in which more resources are allocated to the HP samples that demonstrate more promising performance. Similarly, in [18], an early stop criteria mechanism was incorporated into the algorithm to prevent the model from being evaluated with unfavorable HP samples. Thus, this speeds up the search process as compared to the naive random search approach. Some other hybrid methods were also proposed to speed up the optimization process, among these is the adaptive Bayesian approach, in which dataset sizes were selected adaptively, and the model was evaluated using the Bayesian optimization algorithm [20]. Similarly, the Bayesian optimization approach was combined with the hyperband, thus harnessing the strength of both techniques to effectively search the solution space and to terminate the process [14]. Despite being highly parallel, the lack of consideration of previous experiences in searching for new solutions is a major drawback associated with random search-based approaches. It has been demonstrated that the Gaussian Process (GP) and Bayesian Optimization (BO) are effective methods for optimizing HP configurations. However, they cannot fully take advantage of parallel computing platforms due to their inherently sequential nature. Other approaches that use the power of parallel computing platforms to accelerate HP optimization processes have been reported in [12; 21]. For example, a metaheuristic-based approach called the covariance matrix adaptation evolutionary strategy (CMA-ES) has been proposed to optimize HPs in ML models. Similarly, the PSO-BO algorithm was proposed in [22], which uses the PSO algorithm to optimize the acquisition function of the BO algorithm. Other metaheuristic approaches for hyperparameter optimization are presented in [23; 24]. In addition, a parallel approach referred to as the parallel PSO was proposed in [25] to scale up the HP optimization, while harnessing the strength of the multiple computing resources. The results obtained demonstrated performance improvement over the sequential PSO-BO technique. Other documented approaches entail the use of large population sizes in evaluating the performance of metaheuristics-based optimization of ML model HPs [26; 27]. Various AutoML systems have been proposed in the literature for the practical application of hyperparameter optimization of ML models. The main goal is to automatically optimize the ML pipeline, such as optimizing the ML model selection and hyperparameters. These include Auto-Sklearn [28], Auto-Weka [29], Auto-Keras [15], and Auto-Pytorch [30], which are named after the ML packages they primarily support.