# GlassBoost: A Lightweight and Explainable Classification Framework for Tabular Datasets
## 4 Proposed Method
### Model Compression (Step 3)
The results indicate that a decision tree with a maximum depth of three, using only seven features, is sufficient to create an explainable model with a precision of 0.9320, a recall of 0.9803, and an accuracy of 0.9670 (see row seven of Table 4). Figure 6 illustrates the tree structure of GlassBoost in this case when \(d=7\) and maximum depth = 3. A notable observation from Figure 6 is that not all of the seven most important features are utilised in the decision nodes, while some appear multiple times. This indicates that the decision nodes prioritise features that enhance the model's performance on the training set. Specifically, the features Destination Port and Packet Length Mean are selected multiple times, while the feature Bwd Packets/s is not selected despite having a higher gain value than Destination Port. This suggests that certain features may not improve decision performance if they correlate with higher-ranked features already incorporated into the decision process. The same behaviour is seen when the tree depth is increased. Regarding the color scheme in the decision tree diagram, the two distinct colors represent the two classes in our binary classification task. Additionally, the brightness of each color corresponds to the Gini index at each node, with lighter shades indicating higher impurity. This color-coding approach is consistently applied across all decision tree diagrams presented in this paper. In Figure 6, the decision tree consists of seven leaf nodes. Consequently, the tree can be represented as a rule-based classifier comprising seven classification rules, each corresponding to a single leaf node. These rules can be expressed in the form of IF-THEN  
Figure 5: GlassBoost performance metrics versus the number of features (maximum depth: 3). statements. For example, Equations (25)-(28) illustrate the rules derived from the leftmost leaf of the tree. IF Bwd Packet Length Std \(\leq\) 1485.295, (25)  
AND Packet Length Mean \(\leq\) 5.929, (26)  
AND Destination Port \(\leq\) 84, (27)  
THEN Anomaly Class (Class 1). (28)  
For the next scenario, we limit the maximum depth of the decision trees to four and repeat the same procedure. Figure 7 presents the performance metrics as a function of the number of features for this case. As depicted in Figure 7, utilising a decision tree with a maximum depth of four leads to a substantial improvement in model performance compared to the model with a maximum depth of three. Specifically, by incorporating the eight most influential features, the model achieves a precision of 0.9843, a recall of 0.9792, and an accuracy of 0.9868. Similar to the  
Figure 6: GlassBoost tree structure (maximum depth of three trained on the top seven features). Figure 7: GlassBoost performance metrics versus the number of features (maximum depth: 4). previous analysis, increasing the number of features beyond a certain threshold does not improve performance (in this case, beyond eight). The resulting decision tree for this case (\(d=\)8, maximum depth = 4) is illustrated in Figure 8.  
As shown in Figure 8, the resulting decision tree has ten leaf nodes, implying that an equivalent rule-based classifier would require ten IF-THEN statements. So, increasing the decision tree depth enhances performance but compromises explainability. However, in terms of explaining individual decisions, that increase is marginal. Specifically, for the three-depth tree, six decision states (leaf nodes) have three predicates in their IF-THEN statements, and one decision state has two. Here, six decision states have four predicates in their IF-THEN statements, three have three predicates, and one has two predicates. As the final configuration, an additional experiment is conducted with the maximum tree depth set to five.