## 5 Performance Evaluation
### Experimental Results
#### 5.4.3 Experiment 3  
In this experiment, we first compare DAE+OCSVM with and without the proposed defense (XAI and clustering) in terms of classification performance, as shown in Table 4. Next, we evaluate the robustness of DAE+OCSVM with and without the proposed defense (XAI and clustering) against gradient-based (FGSM and BIM) and optimization-based evasion attacks (CW, ZOO, and DeepFool). The results of these robustness evaluations are provided in Figures 7 and 8 in terms of _DR. It is worth noting that the difference between the DR before (i.e., with no evasion) and after attacks represents the severity of the attacks (i.e., attack success rate)_. The detailed findings from this experiment are as follows:  
* Table 4 compares the classification performance (_ACC, DR, FA, PR, F1_, and _HD_) of DAE+OCSVM with and without the proposed defense. It is evident from this table that the proposed defense improves all the classification performance metrics, which indicates an ability to accurately distinguish between benign and malicious samples. This improvement occurs because the use of clustering allows detectors to train on data with close consumption patterns, which leads to parameters closer to the optimal detector's parameters for individual consumption patterns (i.e., lower level of generalization). Additionally, the use of XAI distinctly separates benign and malicious consumption patterns, leading to easier ET detection. Additionally, Figure 9 shows \begin{table}
\begin{tabular}{l c c c c c c c} \hline \hline
**Dataset** & **ACC** & **DR** & **FA** & **PR** & **F1** & **HD** \\ \hline \multirow{4}{*}{IRISH} & No Defense & 93.61 & 96.74 & 9.51 & 91.05 & 93.81 & 87.23 \\ \cline{2-7}  & Proposed (C1-XAI) & 97.75 & 100.00 & 4.49 & 95.70 & 97.80 & 95.51 \\  & Proposed (C2-XAI) & 97.28 & 100.00 & 5.45 & 94.83 & 97.35 & 95.55 \\  & Proposed (C3-XAI) & 97.60 & 100.00 & 4.80 & 95.42 & 97.65 & 95.20 \\  & Proposed (C4-XAI) & 97.79 & 100.00 & 4.41 & 95.78 & 97.84 & 95.59 \\  & Proposed (C5-XAI) & 97.75 & 100.00 & 4.49 & 95.70 & 97.80 & 95.51 \\ \hline \multirow{2}{*}{PRECON} & No Defense & 85.31 & 86.41 & 15.75 & 83.99 & 85.18 & 70.66 \\ \cline{2-7}  & Proposed (C-XAI) & 92.69 & 96.88 & 11.34 & 89.13 & 92.84 & 85.54 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of DAE+OCSVM with and without the proposed defense (XAI and clustering) in terms of classification performance. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & & **ACC** & **DR** & **PR** & **F1** & **HD** \\ \hline No Evasion & & 93.22 & 97.48 & 89.82 & 93.49 & 86.43 \\ \hline \multirow{2}{*}{CNN-based Attacker} & FGSM & 70.21 & 51.47 & 82.33 & 63.34 & 40.38 \\  & BIM & 67.80 & 46.65 & 80.85 & 59.16 & 35.56 \\ \hline \multirow{2}{*}{FFNN-based Attacker} & FGSM & 71.47 & 53.99 & 83.01 & 65.43 & 42.09 \\  & BIM & 68.46 & 47.96 & 81.28 & 60.32 & 38.87 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The vulnerability of the DAE+OCSVM anomaly detector to evasion attacks. the Precision-Recall (PR) and Receiver Operating Characteristic (ROC) _curves_. These figures provide a visual representation of the performance comparison. They indicate that our proposed detectors achieve higher performance, as evidenced by the values of \(AUC\_PR\) and \(AUC\_ROC\), demonstrating that the proposed detectors enhance ET detection in both the IRISH and PRECON datasets;
* Figure 7 compares the robustness of DAE+OCSVM with and without the proposed defense against CNN-based attacker model in terms of \(DR\). The difference between the \(DR\) before (i.e., with no evasion) and after attacks represents the severity of the attacks or attack success rate (\(ASR\)). In No Defense, the \(DR\) shows a minimal decrease from 97.48% to 46.65% under BIM attack. However, with the proposed defense, the \(DR\) values remain above 90% for all attacks across clusters C1-XAI to C5-XAI. Figure 8 compares the robustness of DAE+OCSVM with and without the proposed defense against the FFNN-based attacker model in terms of \(DR\).