## 4 Experiments
### Performance Evaluation  
#### 4.2.1 HPO for XGBoost  
XGBoost is derived from the gradient boosting framework. It is more efficient because of the algorithm's parallel computation, approximate tree building, efficient handling of sparse data, and memory usage optimization, which make it at least 10 times faster than the existing gradient boosting implementations. XGBoost can handle many tasks, such as regression, classification, and sorting. **Search Space:** We consider the 10 hyperparameters of XGBoost as shown in Table 2. We ran experiments with XGBoost on nine classification datasets, and Table 3 shows the performance. \begin{table}
\begin{tabular}{l l l} \hline \hline
**Name** & **Type** & **Ranges** \\ \hline max\_depth & int & \([1,25]\) \\ learning\_rate & float & \([0.001,0.1]\) \\ n\_estimators & int & \([50,1200]\) \\ gamma & float & \([0.05,0.9]\) \\ min\_child\_weight & int & \([1,9]\) \\ subsample & float & \([0.5,1]\) \\ colsample\_bytree & float & \([0.5,1]\) \\ colsample\_bylevel & float & \([0.5,1]\) \\ reg\_alpha & float & \([0.1,0.9]\) \\ reg\_lambda & float & \([0.01,0.1]\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Hyperparameters of XGBoost. **Results:** We ran each method five times, and the average search time and accuracy on the test set are tabulated in Table 3. It can be seen that RFEPPO outperforms other optimization methods and baseline in terms of accuracy. The accuracy of our method is substantially higher than the baseline, which indicates the importance of HPO. For the search time, hyperband and BOHB spend the least time on all datasets due to the fact that the bandit-based approach would use a very small budget in the initial stage and would quickly discard the poorly performing hyperparameter configurations. However, for the accuracy, both BOHB and hyperband are lower than our method. We believe that BOHB and hyperband may throw away the hyperparameter configurations that really perform well in the initial stage. Compared to RS and TPE, our method can reduce the time by about 27% to 40% on winequality_white, optdigits and Turkive_Student_Evaluation datasets, and reduce the time by about 4% on the Cardioocography dataset. For other datasets, the search time of our method is a little slower. However, the accuracy on test set of our method is better than all other methods. This shows that our method has a clear advantage for datasets with a large sample size and a large number of features. Although the search time of our method is slower for small datasets, the accuracy is higher than other methods. In addition, compared to MBRL-SDP, our method has higher accuracy in the test set. This demonstrates that intrinsic reward can encourage the exploration of novel states and thus quickly find well-performing hyperparameter configurations. To verify the effectiveness of RFEPPO, we visualize the performance curve as the validation accuracy of all methods over a period of wall clock time. Figure 6 compares RFEPPO with all methods on the winequality_white dataset. From the figure, we can see that our method can find good hyperparameter configurations faster and performs better overall compared to MBRL-SDP and RS. As can be seen by the left part of the black line, our method can achieve similar performance to hyperband, and is initially slightly faster than it. Although TPE also converges quickly in the initial stage, the final performance is worse than RFEPPO. This experiment demonstrates that our method not only shortens the search time on large datasets, but also improves the accuracy and reliability of the tasks.