SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Note that such a gap may not only occur when the anomaly detector makes erroneous decisions but also when the anomaly detector correctly detects an attack sample by unreasonable features. For example, the most obvious attribute of a Denial-of-Service (DoS) attack is typically its high rate of forwarding packets in order to overwhelm the victim. However, the provided explanation might suggest the IP address is an important feature for the anomaly detector to make this inference, which is quite doubtful from the perspective of a security expert. In practice, this may be due to "spurious correlation" or "shortcut learning". Specifically, for the DoS example mentioned in our paper, unreliable decisions could be made due to inappropriate testbed settings for the collection of
$^{5}$ D. Arp, E. Quiring, et al., "Dos and Don'ts of Machine Learning in Computer Security", in 31st USENIX Security Symposium, USENIX Security 2022.
training data, such as the DoS attack being launched from one separate host address while all other normal traffic is from other host addresses.
A.4.3 More Hyperparameter Analysis
The robustness of our method against changes in hyperparameters also needs to be evaluated. Sensitivity analysis analysis us to identify which parameters are most influential, thereby concentrating on tuning those to achieve optimal performance. In our study, we perform a sensitivity analysis on the hyperparameters: Maximum Iteration, Auxiliary Explorer Sample Size  $(N_{s})$ , Distribution Similarity Threshold  $(\epsilon)$ , and Significance Threshold  $(\delta)$ .
Maximum Iteration represents the maximum number of iterations that the algorithm will run. In our analysis, as shown in Table 9, this parameter's variation from 0.01 to 0.5 has a negligible effect on all the measured indicators. This displays the algorithm's resilience and robustness to changes in this hyperparameter, indicating that the model converges relatively rapidly.
Auxiliary Explorer Sample Size ( $N_{s}$ ) represents the number of samples gathered to assist the model's exploration phase. The results indicate that our model effectively makes use of the available samples because an increase in  $N_{s}$  has little to no effect on the majority of the metrics. However, a slight increase in fidelity, correctness, and accuracy is observed at larger sample sizes, indicating that providing more samples can marginally enhance the model's performance.
Distribution Similarity Threshold  $(\epsilon)$  dictates how similar the proposed and baseline distributions should be for the tree's branches to accept the proposed distribution. Interestingly, whereas other metrics practically remain constant as epsilon goes from 10 to 150, fidelity, correctness, and accuracy show a slight uptick. This suggests that our model exhibits an adaptable behavior to changes in distribution similarity thresholds.
Significance Threshold  $(\delta)$ , which determines the minimum difference that should be deemed noteworthy, also shows a negligible effect on all the performance indicators. This shows that the model is not sensitive to changes in  $\delta$  and supports the robustness of the proposed approach.
One noticeable aspect across all parameters is the TPR and TNR values. The TPR is consistently at its maximum, showcasing the system's excellent ability to identify positive instances. TNR also remains high, demonstrating the system's effectiveness in accurately identifying negative instances.
This sensitivity analysis underlines the robustness and adaptability of our proposed method. Despite varying hyperparameters, our method consistently delivers high performance, emphasizing its potential as a versatile solution in diverse real-world scenarios. However, care should be taken to appropriately tune these parameters according to the specifics of the given context.
Future work might explore other influential hyperparameters and conduct a similar sensitivity analysis. Moreover, utilizing methods like automatic hyperparameter tuning or evolutionary algorithms can lead to more efficient and optimal configuration settings.
Table 9: Performance of different hyperparameters.
Metric	Maximum Iteration	Auxiliary Explorer Sample Size	Distribution Similarity Threshold	Significance Threshold
0.01	0.05	0.1	0.5	50	100	150	350	10	50	100	150	0.001	0.005	0.01	0.1
Fidelity	0.9996	0.9996	0.9996	0.9995	0.9996	0.9996	0.9996	0.9995	0.9996	0.9998	0.9998	0.9998	0.9996	0.9996	0.9996	0.9995
Robustness	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	0.9995
Correctness	0.9992	0.9992	0.9992	0.9992	0.9993	0.9992	0.9992	0.9992	0.9992	0.9995	0.9996	0.9996	0.9992	0.9992	0.9992	0.9992
Accuracy	0.9992	0.9992	0.9992	0.9992	0.9993	0.9992	0.9992	0.9992	0.9992	0.9995	0.9996	0.9996	0.9992	0.9992	1.0000	0.9995
TPR	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	1.0000	2.0000	1.0000
TNR	0.9844	0.9845	0.9849	0.9842	0.9853	0.9846	0.9845	0.9841	0.9844	0.9909	0.9925	0.9920	0.9850	0.9847	0.9848	0.9841
A.5 Limitations
Though our method achieves global explanation with high fidelity, it still has several limitations. First, our method works well with tabular data but might be inapplicable to raw image data. It is because our method treats every dimension of the feature space as a semantic feature, such as the average packet size of a network connection, and extracts rules from each dimension of the feature space. In
contrast, raw image data are tensors of pixels. Their high-level semantics cannot directly derive from each of the pixels but usually need a deep model with spatial awareness (e.g., CNN, ViT) to extract feature maps, which are inconsistent with our method. Hence, this issue may limit the transferability of the proposed method to other domains. Nonetheless, due to the typical trust in expert knowledge over deep models in security domains, most security applications still rely on sophisticated feature engineering and use data representations with explicit semantics, suggesting that our method remains general in the field of security.
Second, recall that the rules extracted by our method are axis-aligned, which can be interpreted as a certain feature over/under a threshold and are human-understandable. Though this format of rules significantly promotes interpretability, it may limit its degree of fitting to the decision boundary of the original model, which can be of various shapes in the high-dimensional feature space for different models. Though our IC-Tree has mitigated this issue by splitting the distribution of normal data into multiple compositional distributions, which are more compact and more likely to be fitted using axis-aligned rules, there is little guarantee that our method cannot encounter underfitting if the decision boundary of the original model is extremely irregular. It should be clarified that this is a common limitation for all the global explanation methods that employ axis-aligned rules or decision trees as the surrogate expression. To this end, we are also exploring other surrogate models and algorithms that can further balance the interpretability and fitting ability.
Lastly, as we mentioned above, we believe that the experiments on "human understanding and trust" can be significantly strengthened by introducing security practitioners to participate in a use test, finding if the interpretation provided by the proposed method is consistent with their expert knowledge of judging an anomaly. Currently, we are collaborating with the Tencent Security Platform Department, aiming to rectify this limitation by accounting for the opinions of real practitioners.