# GlassBoost: A Lightweight and Explainable Classification Framework for Tabular Datasets
## 4 Proposed Method
### Model Compression (Step 3)  
The model compression process is illustrated in Figure 4. The approach leverages gain scores to identify and select the most significant features. The selected features compress a complex XGBoost model into a simplified, more interpretable decision tree. The model compression reduces the number of contributing features and streamlines the decision-making structure, replacing a complex ensemble of multiple boosting trees with a single decision tree. Although XGBoost is highly effective in handling tabular data, its ensemble nature, consisting of multiple decision trees, makes the prediction process challenging to  
Figure 3: Sorted gain scores. Figure 2: Feature importance scores. interpret. Therefore, instead of using an XGBoost model for classification, we use the gain scores to identify the most important features. We select the \(d\) most important features, sort them, and use them to train a decision tree. We now provide a detailed explanation of the model compression process. Suppose that we are given a dataset of \(N\) samples where each sample \(i,1<i<N\), is presented by the feature vector \(X_{i}=[x_{i1},x_{i2},\ldots,x_{in}]\) from the original feature space and \(n\) is the number of features of the feature vector \(X_{i}\). We define a dimensionality reduction transform \(T_{d}(. )\) from the original feature space onto the reduced feature space so that \(T_{d}(X_{i})=Y_{i}\), where \(Y_{i}=[y_{i1},y_{i2},\ldots,y_{id}]\) consists of the \(d\) features from \(X_{i}\) with the highest gain scores sorted in descending order based on their respective gain scores. The number of features in \(Y_{i}\) should be much smaller than that in \(X_{i}\) (\(d<<n\)). We can represent a set of \(M\) feature vectors from the original dataset by the matrix \(\mathbf{X}\), a matrix of \(M\) rows (samples) each containing \(n\) features. Applying \(T_{d}(. )\) on rows of \(\mathbf{X}\) yields a data matrix \(\mathbf{Y}_{d}\) in the reduced feature space (see (24)) with the same number of rows (samples) but with a much smaller number of columns (features). Figure 4: Compressing XGBoost into a decision tree. \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Feature** & **Gain** & **Cover** & **Weight** \\ \hline Bwd Packet Length Std & 1.000000 & 0.619524 & 0.400000 \\ Packet Length Mean & 0.874347 & 0.861828 & 0.191304 \\ Bwd Packets/s & 0.186786 & 0.204659 & 0.191304 \\ Destination Port & 0.099629 & 0.127199 & 1.000000 \\ Flow IAT Mean & 0.092572 & 0.133998 & 0.078261 \\ URG Flag Count & 0.067601 & 0.112332 & 0.000000 \\ Active Std & 0.061287 & 0.886808 & 0.086957 \\ Init\_Win\_bytes\_backward & 0.060260 & 0.212840 & 0.504348 \\ Fwd Header Length & 0.054233 & 0.115591 & 0.078261 \\ Bwd IAT Std & 0.054226 & 1.000000 & 0.043478 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Top 10 features ranked by gain scores with corresponding cover and weight scores. to 0.9165, indicating a more balanced approach between true positive and true negative classifications. This trade-off highlights the impact of feature inclusion on model behaviour, particularly in scenarios where high recall is critical, such as intrusion detection. No further improvement is observed beyond a certain point (\(d>7\)). This can be attributed to the structure of the decision tree. A maximum tree depth of three allows for, at most, seven splitting nodes. Consequently, any additional features beyond this threshold do not contribute to the decision-making process. In other words, these later features are not selected for the decision nodes because the earlier, more discriminative features have already been selected. Figure 5 illustrates the performance metrics for this scenario across a broader range of selected features (\(1\leq d\leq 40\)), visually representing how these metrics evolve.