## 4 Results
### Experimental Setup  
**Computing Resources:** Our experiments utilized a high-performance computer (HPC) equipped with four NVIDIA A100 GPUs and 64 GPU-accelerated nodes (NVIDIA, Santa Clara, CA, USA), each with 256 GB of memory, alongside a 64-core AMD EPYC 7713 processor (AMD, Santa Clara, CA, USA) (2.0 GHz, 225 watts). This setup delivered a peak performance of about 7 petaFLOPs.  
**Coding Tools:** For our development, we chose the Python programming language to take advantage of various open-source resources and maintain transparency in our implementation. Our toolkit included Keras and ScikitLearn for AI modeling. Additionally, we integrated XAI toolboxes, such as the following:  
**(a)**: **SHAP [50]:** Employed for generating feature explanations, this tool was instrumental in providing both local and global insights into our AI models for intrusion detection.
**(b)**: **LIME [51]:** We used LIME to develop surrogate models that approximate the decisions of our AI models on individual samples, helping in crafting detailed local explanations.
**(c)**: **Other toolboxes:** Tools like Pandas and Matplotlib were also utilized for data manipulation and visualization tasks.  
\begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline \multicolumn{10}{c}{(a) Basic statistics of datasets} \\ \hline
**Dataset** & **No. of Labels** & \multicolumn{2}{c}{**No. of Features**} & \multicolumn{2}{c}{**No. of Samples**} \\ \hline
**CIEDS-2017** & 7 & 78 & 2,775,364 & \\ \hline
**RoEduNet-SIMARGL2021** & 3 & & 29 & & 31,433,875 & \\ \hline
**NSL-KDD** & 5 & & 41 & & 148,517 & \\ \hline \multicolumn{10}{c}{(b) Distribution of samples among different attack types} \\ \hline
**Dataset** & **Normal** & **DoS** & **PortScan** & **Brute** & **Web** & **Attack** & **Bot** & **Infiltration** & **Probe** & **U2R** & **R2L** \\ \hline
**CIEDS-2017** & 84.442\% & 9.104\% & 5.726\% & 0.498\% & 0.157\% & 0.071\% & 0.001\% & - & - & - \\ \hline
**RoEduNet-SIMARGL2021** & 62.20\% & 24.53\% & 13.27\% & - & - & - & - & - & - & - \\ \hline
**NSL-KDD** & 48\% & 35\% & - & - & - & - & - & 10.05\% & 0.47\% & 6.48\% \\ \hline \hline \end{tabular}
\end{table}
Table 5: A summary of the basic statistics and distribution of samples among different attack types for the three datasets.  
**AI Models:** We assessed seven widely used AI classification models: a deep neural network (DNN), random forest (RF), AdaBoost (ADA), K-nearest neighbor (KNN), support vector machine (SVM), Multi-Layer Perceptron (MLP), and Light Gradient-Boosting Machine (LightGBM). These models were evaluated to test both the opaque nature of black-box AI and the transparency features of our XAI framework in network intrusion detection.  
**Hyperparameters:** Specific details on the hyperparameter settings for each model are available in Appendix C.  
**Metrics for Black-box AI:** We evaluated the black-box AI models using various metrics, such as accuracy (Acc), precision (Prec), recall (Rec), F1 score (F1), the Matthews correlation coefficient (Mcc), balanced accuracy (Bacc), and the area under the ROC curve (AucRoc). These metrics provide a thorough analysis of the AI models' performance across various benchmarks.  
**Explainability Measures:** Our evaluation of XAI models included several explainability measures: global summary plots (for assessing overall feature influence), global beeswarm plots (to examine the correlation between features and AI decisions), local explanations by LIME (which details the reasoning behind AI decisions for individual instances using surrogate models), local explanations by SHAP (which uses Shapley values to explain decisions for individual instances), feature importance per attack type (highlighting key features indicative of specific attack types), and overall feature rank per dataset (identifying common important features across different models). These metrics are crucial for evaluating the effectiveness of our explanations and comparing different explanation methods.