## 4 Experiments
### Experimental Setup  
**Datasets:** We validated our method on 11 datasets, 9 of which are tabular data to verify the performance of RFEPPO tuning the hyperparameters of XGBoost, and the remaining two image datasets to verify the performance tuning the hyperparameters of CNN. Table 1 shows the details of the datasets. The tabular datasets are from the UCI Machine Learning Repository [31] and openML [32]. These datasets come from different fields, including computer, medicine, education, life, and physics. The MNIST dataset [33] and Fashion MNIST dataset [34] are classic image classification datasets in the field of ML. Both of them consist of \(60,000\) training samples and \(10,000\) test samples, each of which is a \(28\times 28\) pixel grayscale image. Fashion MNIST is identical to the original MNIST in training set/test set division, size and format. For the tabular datasets, we divide each dataset into training, validation and test set by a ratio of \(6:2:2\); for the image datasets, we follow the original split ratio and then use the last 5000 samples of the training set as the validation set. **Comparison Methods:** In this paper, we compared our approach with the following optimization algorithms: RS [6], TPE [5], hyperband [1], BOHB [12], and the default configuration of XGBoost (baseline). In this paper, we implemented these algorithms with the Hyperopt and HpBandSter Libraries. In addition, we also compared our method with the MBRL-SDP model proposed in the [15]. RS randomly selects samples within the search range. By random sampling, the global optimum or its approximation can also be searched with high probability if the set of samples is sufficiently large.TPE is a sequential model-based optimization (SMBO) method that tries to build a probabilistic \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Default Task** & **Datasets** & **\#Instances** & **\#Attributes** \\ \hline \multirow{4}{*}{Multi Classification} & winequality\_white & 4898 & 11 \\  & optdigits & 5620 & 64 \\  & Turkivé\_Student\_Evaluation & 5820 & 28 \\  & balance\_scale & 625 & 4 \\  & Cardiotocography & 2126 & 21 \\ \hline \multirow{4}{*}{Binary Classification} & monks\_1 & 556 & 6 \\  & DR\_Debrecen & 1151 & 19 \\  & churn & 5000 & 20 \\  & socmob & 1156 & 5 \\ \hline \multirow{2}{*}{Image Classification} & MNIST & 70,000 & \(28\times 28\times 1\) \\  & Fashion\_MNIST & 70,000 & \(28\times 28\times 1\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Details of datasets. model of the function at each step and selects the most promising parameters for the next step. Hyperband attempts to explore as many configurations as possible with limited resources and returns the most probable configuration as a final result. The weakness of hyperband is generating new configurations randomly without leveraging finished trials. BOHB is a follow-up work to hyperband, which generates a certain percentage of new configurations by constructing multiple TPE models to take advantage of the finished trials. **Evaluation metrics:** The performance of HPO methods are evaluated using the following metrics:  
* Time: average search time required to search 200 episodes per method. * Accuracy: average accuracy of each method in the test set. **Details:** As presented in Section 3.2, the agent consists of an input FC layer, three LSTM layers with hidden state size of 128, and an output FC layer. The learning rate of the agent is set to 0.001. The rest of the settings are configured using the default configuration of the PPO algorithm. Before using the surrogate model RF, we adopted grid search to find the best hyperparameters for RF. The accuracy of the validation set is collected for the first 100 episodes, and then the combination of hyperparameters and accuracy from these 100 episodes are used as samples to train the RF: The KL distance threshold between policies is set to 0.1. For the curiosity model, the embedding layer captures state information and extends the state to 32 dimensions.