# Multi-Granularity User Anomalous Behavior Detection
## 5 Experiments
### Anomaly Detection on Different Behavior Data Ratios
#### 5.3.1 Results on Behavior Level (CAD)  
We extracted the behavioral data of 70 anomalous users and a randomly selected subset of normal users from the CERT R4.2 dataset. We analyzed the impact of using \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Model** & **PRE (\%)** & **DTR (\%)** & **F1 (\%)** & **ACC (\%)** & **FPR (\%)** \\ \hline S-LSTM [6] & NA & 96.00 & 95.45 & 99.00 & 2.00 \\ ITDBERT [10] & 93.00 & 91.87 & 92.43 & NA & NA \\ SPYRAPTOR [24] & 91.00 & 89.00 & 90.00 & 99.00 & NA \\ SeqA-ITD [25] & 95.79 & 95.64 & 95.63 & NA & NA \\ RAP-Net [26] & 95.71 & 96.01 & 95.86 & NA & NA \\ LSTM-Autoencoder [27] & 97.00 & 92.00 & 94.00 & 90.60 & 9.00 \\
**MG-UABD** (ours) & **99.99** & **99.99** & **99.99** & **99.99** & 9.69 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Comparison results of anomaly detection on the behavior level. The bolded values are the optimal results, while the underlined values are the suboptimal results. different proportions of training data on the anomaly detection performance within the fine-grained anomaly detection module, for each individual user. The aggregated results across all users were considered as the overall detection outcome. In Figure 3, moving from left to right, the changes in the percentage of TN (True Negatives), TP (True Positives), and FN (False Negatives) samples in the anomaly detection results are depicted for training data proportions set at 20%, 40%, 60%, and 80% (Due to the model's accurate prediction of normal behavior (TP) occupying a significant proportion, which affects the visualization of other sample percentages and is not the focus of this analysis, TP is not shown). From the figure, it can be observed that as the proportion of the training set increases, the percentage of TNs notably rises, while the percentages of FP (False Positives) and FNs decrease significantly. This phenomenon suggests that with a larger training set, the model's capability to uncover anomalous samples improves, detecting more anomalies. Simultaneously, the percentages of misclassified anomalous and normal samples decrease. This observation indicates that a larger training dataset enhances the model's capability to perform anomaly detection. We analyzed the changes in various anomaly detection metrics for both the fine-grained anomaly detection module and the MG-UABD system as a whole, as the proportion of the training set increased. As illustrated in Figure 4a,b, the performance across all metrics improved consistently as the training set proportion grew, for both the fine-grained anomaly detection module and the entire MG-UABD system. Specifically, Figure 4a shows that when the training set proportion was 20%, PRE, DTR, F1, and ACC were at their lowest levels. As the proportion of the training set increased, the marginal benefit of improvement in these metrics decreased. Among the four metrics, PRE, DTR, and F1 each increased by approximately 0.02%, while ACC, being a composite metric of PRE and DTR, increased by 0.04%. Additionally, the False Positive Rate (FPR) decreased from about 20% to around 10%. Combining the results of the coarse-grained anomaly behavior detection, we obtain the overall anomaly detection results for MG-UABD, as shown in Figure 4b. The trend of changes in the metrics as the proportion of training data varies is similar to that of the fine-grained module. However, due to the inclusion of the coarse-grained anomaly detection results, which serve to moderate the overall effect, the growth rates of the metrics are diluted. PRE, DTR, and F1 increase by 0.002%, 0.0015%, and 0.001% respectively, while ACC grows by 0.004%. The False Positive Rate (FPR) decreases from about 20% to approximately 9%. Figure 3: Fan chart of the sample distribution for user behavior detection. The experimental results indicate that an increase in training data benefits the model's anomaly behavior detection performance, but the marginal benefit of improvement in the metrics gradually diminishes.