## 2 Materials and Methods
### Catboost Model Based on Optimisation of Optuna Hyperparameters  
CatBoost is an advanced machine learning algorithm based on Gradient Boosting Decision Trees (GBDT). It effectively addresses the challenges posed by large feature spaces and class imbalance through gradient optimization techniques. The algorithm utilizes a fully symmetric tree as its base learner, as illustrated in Figure 6[33]. Unlike a typical decision tree, a fully symmetric tree ensures that internal nodes at the same depth employ identical features and feature thresholds for splitting. Consequently, a fully symmetric tree can be represented as a decision table with \(2^{d}\) entries, where \(d\) represents the number of  
Figure 4: Comparison of data distribution before and after UNSW-NB15 balancing. Figure 5: Comparison of data distribution before and after CICIDS2017 balancing. Figure 3: Comparison of data distribution before and after NSL-KDD balancing. levels in the tree [34]. This balanced structure enhances processing speed and improves feature handling compared to a standard decision tree. In the domain of network intrusion detection, certain classes (intrusions) often suffer from underrepresentation. When confronted with large-scale datasets in network intrusion detection, employing the CatBoost algorithm as a classifier offers inherent advantages in handling high-dimensional data, addressing the class imbalance, managing classification features, facilitating automatic feature scaling, and ensuring computational efficiency. CatBoost excels in mitigating sample imbalance through the utilization of weighted loss functions and class weight adjustment, thereby enhancing classification accuracy for minor classes. It mitigates the risk of overfitting by introducing random statistical alignment during model training, improving generalization performance. By adeptly tackling the challenges associated with network intrusion detection datasets and delivering efficient classification capabilities, CatBoost proves to be a valuable choice for this task. CatBoost is built upon the foundation of a gradient-boosting tree. However, it deviates from the conventional approach by eschewing a greedy target statistics-based method for splitting nodes when handling category features. Instead, CatBoost considers the prior distribution term during the calculation of node gain. This approach effectively mitigates the impact of low-frequency features and noise in the category variables on the construction of decision trees. \[\text{x}_{i,k}=\frac{\sum_{j=1}^{P-1}\left[x_{\sigma_{j,k}}=x_{\sigma_{p,k}} \right]\times Y_{j}+a\times P}{\sum_{j=1}^{P-1}\left[x_{\sigma_{j,k}}=x_{ \sigma_{p,k}}\right]+a} \tag{7}\]  
where \(\sigma_{j}\) is the \(j\) th data; \(\text{x}_{i,j}\) denotes the \(,k\) th column of discrete features of the \(i\) th row of data in the training set; \(a\) is a prior weight; and \(P\) is the prior distribution term. Optuna is a framework for automated hyperparameter optimization, which efficiently explores the hyperparameter space to discover the optimal combination of hyperparameters [35]. When it comes to the CatBoost algorithm, there are several hyperparameters that can be fine-tuned, including learning rate, number of trees, and depth. Manually tuning these hyperparameters can be time-consuming and requires expertise in the field. By leveraging Optuna, the model's robustness can be enhanced through the optimization of hyperparameters, resulting in a selection of hyperparameter combinations that exhibit excellent performance across different environments [36]. The utilization of Optuna to optimize CatBoost facilitates improved model performance, generalization, and interpretability, ultimately yielding superior hyperparameters for intricate tasks like intrusion detection. This process unfolds through the following steps:  
1. Define hyperparameter space:  
Before employing Optuna for hyperparameter optimization, it is essential to establish the search space for the hyperparameters.