# Multi-Granularity User Anomalous Behavior Detection
## 5 Experiments
### Overall Results of Anomalous Behavior Detection (CAD)  
MG-UEBA employs a combination of coarse-grained and fine-grained modules for the detection of anomalous user behavior. The code implementation for MG-UABD is available at [[https://github.com/cao-yuu/MG-UABD](https://github.com/cao-yuu/MG-UABD)]([https://github.com/cao-yuu/MG-UABD](https://github.com/cao-yuu/MG-UABD)) (accessed on 17 December 2024). Table 5 presents the detection outcomes of anomalous users and behaviors by the MG-UABD system. At the user level, out of a total of 1000 users, only 2 anomalous users were not detected and were instead classified as normal users. At the behavior level, from 6,554,045 behavior samples under examination, 141 anomalous behavior samples were misclassified as normal, thus remaining undetected, while 221 normal behavior samples were incorrectly categorized as anomalous. The majority of users and behavior samples were correctly identified by MG-UABD.  
The experimental results shown in Table 6 illustrate the performance differences between MG-UABD and other models in the task of anomaly detection at the behavior level. The baseline models listed in the table all conducted anomaly detection experiments on the CERT R4.2 dataset, and the results presented are directly sourced from their respective original publications. Unlike these models, our approach to coarse-grained anomaly detection did not involve random splitting of the training and testing sets. Instead, we used the first 25 weeks of user behavior data as the training set and the remaining data as the test set. This division method increases the difficulty of the anomaly detection task. However, as can be seen from the table, despite the more stringent testing conditions, MG-UABD outperforms all baseline models in all metrics except for FPR (False Positive Rate).  
\begin{table}
\begin{tabular}{l r r r r} \hline \hline
**Confusion Matrix** & **TP** & **TN** & **FP** & **FN** \\ \hline user level & 930 & 68 & 2 & 0 \\ behavior level & 6,552,369 & 1314 & 141 & 221 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Confusion matrix of MG-UABD on anomalous and normal user behavior detection.  
Compared to the second-best model, MG-UABD achieved improvements of 3.1%, 4.1%, 4.3%, and 1.0% in PRE (Precision), DTR (Detection Rate), F1 (F1 Score), and ACC (Accuracy), respectively, calculated as follows:  
\[PRE\uparrow=3.1\%=\frac{(99.99-97.00)}{97.00}, \tag{6}\]  
compared with LSTM-Autoencoder [27]:  
\[DTR\uparrow=4.1\%=\frac{(99.99-96.01)}{96.01}, \tag{7}\]  
compared with RAP-Net [26]:  
\[F1\uparrow=4.3\%=\frac{(99.99-95.86)}{95.86}, \tag{8}\]  
compared with RAP-Net [26]:  
\[ACC\uparrow=1.0\%=\frac{(99.99-99.00)}{99.00}, \tag{9}\]  
compared with SPYRAPTOR [24] and S-LSTM [6].  
Due to the reorganization of the dataset according to chronological order, MG-UABD's FPR is slightly higher than that of other models, indicating that a small number of anomalous behavior samples were not detected.  
This section of the experiment demonstrates that the MG-UABD, employing a two-stage approach of coarse-grained anomaly detection followed by fine-grained anomaly detection, significantly enhances the effectiveness of anomaly detection. It should be noted that the results shown in the table represent the models' detection of all samples of both normal and anomalous user behaviors. Since normal behavior samples constitute a vast majority, the models tend to predict this part of the sample accurately, hence the metrics are all above 90%. In the following sections, we will conduct a detailed analysis of the models' ability to detect anomalous users and anomalous behavior samples.