# GlassBoost: A Lightweight and Explainable Classification Framework for Tabular Datasets
## 4 Proposed Method
### Calculating Feature Importance Scores (Step 2)  
Feature importance scores quantify the contribution of each feature to a model's predictions. These scores help identify the most relevant features, improving interpretability by reducing dimensionality (towards smaller, scored decision trees in GlassBoost).  
After training an XGBoost model, three feature importance scores (weight, gain, and cover) can be extracted for each feature. Weight counts how often a feature is used for splits, while cover measures the average number of samples those splits affect. The most crucial metric is the gain score, which quantifies the average improvement in the model's objective function (e.g., loss reduction) when a feature is used for splitting. Gain is calculated as the difference in loss before and after a split, averaged across all occurrences of the feature in  
\begin{table}
\begin{tabular}{l l} \hline \hline
**Retained** & **Discarded** \\ \hline Fwd Header Length & Fwd Header Length \\ SYN Flag Count & Fwd PSH Flags \\ Avg Fwd Segment Size & Fwd Packet Length Mean \\ Subflow Fwd Bytes & Total Length of Fwd Packets \\ Subflow Bwd Packets & Total Backward Packets \\ Total Fwd Packets & Subflow Fwd Packets \\ \hline \hline \end{tabular}
\end{table}
Table 2: Identified duplicate features in the dataset.  
the boosting trees. A higher gain indicates that a feature contributes more to improving model performance.  
When training an XGBoost model, every time a feature is used to split a node, the gain is computed for that split according to Equation (23). XGBoost sums the gain values across all occurrences of that feature in all boosting decision trees [31]. Readers are referred to Section 3.2 for further information on gain score calculation.  
We trained the XGBoost model on the dataset using 70% of samples for training and the remaining 30% for validation. The loss function values were monitored at each boosting step during training to prevent overfitting. The training samples were selected randomly, and the maximum number of boosting trees was limited to 50. The boosting tree depth was limited to 4, and the objective function binary:logistic was used which is standard objective function for binary classification tasks [31]. Regarding the number and depth of the boosting trees, we should emphasise that our objective is not to construct a model for deployment, but rather to construct a sufficiently accurate model capable of capturing meaningful patterns in the training data without overfitting. This model is primarily used as a tool for extracting gain scores to identify important features. Based on the performance metrics obtained under this configuration, the XGBoost model has been trained effectively, demonstrating excellent results on the test dataset, with an accuracy of 0.9960, a precision of 0.9921, and a recall of 0.9970. These results indicate that the XGBoost model has been trained effectively and that the computed gain scores are reliable.  
Figure 2 shows the feature importance scores: gain, weight, and cover. It is worth noting that the feature importance scores have been scaled to the range [0, 1]. Additionally, the features on the \(x\)-axis are arranged in descending order based on their gain score.  
Figure 1: Covariance matrix heatmap of the features after the preprocessing step.  
Figure 3 only presents the gain scores, while Table 3 lists the top 10 most influential features based on their gain scores, corresponding weight and cover values. Since the gain scores of some features are minimal (approximately zero), the \(x\)-axes for both Figures 2 and 3 were not extended to include all the features.