# Tree-Based Classifier Ensembles for PE Malware Analysis: A Performance Revisit
## 3 Materials and Methods
### Tree-Based Ensemble Learning
CatBoost, unlike other GBDT algorithms, is able to accommodate gradient bias and prediction shift to increase the accuracy of predictions and generalization ability of large datasets. In addition, CatBoost is comprised of two essential algorithms: ordered boosting, which estimates leaf values during tree structure selection to avoid overfitting, and a unique technique for handling categorical data throughout the training process. An implementation of CatBoost in \(R\) is employed in this paper, whereas the search space of each hyperparameter is considered as follows. Depth = {1, 2,..., 10}, learning rate = {0.03, 0.001, 0.01, 0.1, 0.2, 0.3}, l2 leaf regularization = {3, 1, 5, 10, 100}, border count = {32, 5, 10, 20, 50, 100, 200}, and boosting type = {\"Ordered\", \"Plain\"}. The final learning parameters of CatBoost for each malware dataset are given in Table 4. \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Hyperparameter** & **BODMAS** & **Kaggle** & **CIC-MalMem-2022** \\ \hline _Native_ & & & \\ \hline Maximum depth & 11 & 19 & 19 \\ eta & 0.2 & 0.3 & 0.1 \\ Subsample & 0.6 & 0.8 & 0.6 \\ Column sample by tree & 0.7 & 0.5 & 0.6 \\ \hline \(H2O\) & & & \\ \hline Maximum depth & 24 & 23 & 26 \\ Sample rate & 0.52 & 0.94 & 0.99 \\ Column sample rate & 0.42 & 0.62 & 0.6 \\ Column sample rate per tree & 0.6 & 0.25 & 0.5 \\ Minimum rows & 2 & 2 & 2 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The final learning parameters of XGBoost used for each dataset after performing a _random search_. * [(iii)] Gradient boosting machine [25] GBM is the first implementation of GBDT to utilize a forward learning technique. Trees are generated in a sequential manner, with future trees being dependent on the results of the preceding trees. Formally, GBM is achieved by iteratively constructing a collection of functions \(f^{0},f^{1},\ldots,f^{t}\), given a loss function \(\Omega(y_{i},f^{t})\). We can optimize our estimates of \(y_{i}\) by discovering another function \(f^{t+1}=f^{t}+h^{t+1}(\mathbf{x})\), such that \(h^{t+1}\) reduces the estimated value of the loss function. In this study, we adopt GBM implementation in H2O, whereas the hyperparameters' search space is specified as follows. Maximum depth = {1, 3, 5,..., 29}, sample rate = {0.2, 0.3,..., 1}, column sample rate per tree = {0.2, 0.21, 0.22,..., 1}, column sample rate change per level = {0.9, 0.91,..., 1.1}, number of bins = 2{4,5,...,10}, and minimum rows = {0, 1,..., log\({}_{2}\times\) number of rows \(-\) 1}. Table 5 shows a list of all the final GBM hyperparameters that were used on each malware dataset. * [(iv)] LightGBM [26] LightGBM is an inexpensive gradient boosting tree implementations that employs histogram and leaf-wise techniques to increase both processing power and prediction precision. The histogram method is used to combine features that are incompatible with each another. Before generating a \(n\)-width histogram, the core idea is to discretize continuous features into \(n\) integers. Based on the discretized values of the histogram, the training data are scanned to locate the decision tree. The histogram method considerably reduces the runtime complexity. In addition, in LightGBM, the leaf with the greatest splitting gain was found and then divided using a leaf-by-leaf strategy. Leaf-wise optimization may result in overfitting and a deeper decision tree. To ensure great efficiency and prevent overfitting, LightGBM includes a maximum depth constraint to leaf-wise. In this study, we employed a LightGBM implementation in \(R\) with the following hyperparameter search space; Maximum bin = {100, 255}, maximum depth = {1, 2,..., 15}, number of leaves = 2\({}^{(1,2,...,15)}\), minimum data in leaf = {100, 200,..., 1000}, learning rate = {0.01, 0.3, 0.01}, lambda 1I = {0, 10, 20,..., 100}, lambda 1Z = {0, 10, 20,..., 100}, lambda 1Z = {0, 10, 20,..., 100}, feature fraction = {0.5, 0.9}, bagging fraction = {0.5, 0.9}, path smooth = {1 \(\times\) 10\({}^{-8}\), 1 \(\times\) 10\({}^{-3}\)}, and minimum gain to split = {0, 1, 2,..., 15}.