# Streaming-Based Anomaly Detection in ITS Messages
## 3 Results  
This section presents the results obtained from experimenting with anomaly detection approaches on CAMs. The algorithms were written in Python using the Python Streaming Anomaly Detection (PySAD) framework [52]. This allows us to integrate batch processing algorithms from the Python outlier detection (PyOD) framework [53] and apply them to streaming data using a sliding window. The first series of studies was designed to see if the proposed ELSCP employing Pearson correlation outperformed LSCP. We wanted to know which model best estimated anomalous cases and what effect variations in window size had on model performance. Several experiments were performed with varying window sizes. Both algorithms employed two base detectors, HBOS and LOF. Table 2 summarises the parameters used in the experiments.  
Table 3 summarises the experimental results. Based on the data, it is evident that the improvements to ELSCP resulted in better AUC-ROC and AUCPR performance. Figures 5 and 6 indicate that the performance of both algorithms for AUC-ROC and AUCPR improves gradually with increasing the window size.  
The second series of tests was designed to assess the ELSCP performance by examining the true positive and false positive rates using a receiver operating characteristic (ROC) curve. The performance was also examined in terms of precision and recall. The baseline AUC-ROC is usually set at a value of 0.5, which suggests no discrimination; 0.7 to 0.8 is considered acceptable; 0.8 to 0.9 is considered excellent; and more than 0.9 is considered outstanding [54]. ELSCP was trained with 67% of the data, and the prediction performance was tested with 33% of the data. ELSP achieved 0.97 area under the ROC curve in both the true normal data (class 0) and true anomalies (class 1), which is outstanding performance  
\begin{table}
\begin{tabular}{l l} \hline \hline
**Parameters** & **Values** \\ \hline Window sizes & 50, 100, 200, 300, 400, 500, 600 \\ \hline Sliding window size & 50 \\ \hline Initial window (training set) & 1000 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Experimental parameters.  
\begin{table}
\begin{tabular}{l l l l l} \hline \hline  & \multicolumn{2}{c}{**AUC-ROC**} & \multicolumn{2}{c}{**AUCPR**} \\ \hline
**Window Size** & **LSCP** & **ELSCP** & **LSCP** & **ELSCP** \\ \hline
50 & 0.7408 & 0.7794 & 0.1793 & 0.2157 \\ \hline
100 & 0.7763 & 0.8065 & 0.2126 & 0.2496 \\ \hline
200 & 0.8203 & 0.8453 & 0.2525 & 0.3125 \\ \hline
300 & 0.8706 & 0.8977 & 0.2989 & 0.3841 \\ \hline
400 & 0.8833 & 0.9138 & 0.3145 & 0.4208 \\ \hline
500 & 0.8910 & 0.9247 & 0.3206 & 0.4331 \\ \hline
600 & 0.8917 & 0.9277 & 0.3206 & 0.4373 \\ \hline Average & 0.8392 & 0.8707 & 0.2713 & 0.3504 \\ \hline \hline \end{tabular}
\end{table}
Table 3: ELSCP AUC-ROC and AUCPR performance results for window size variation.  
Figure 8: ELSCP precision–recall performance evaluation.  
Figure 7: ELSCP ROC curves performance evaluation.  
In the third set of tests, we endeavoured to determine the anomaly detection method that best estimates the number of anomalous events in C-ITS data. We used the data to analyse the performance of MCD, IForest, RRCF, exact-STORM, LSCP, and ELSCP algorithms in terms of AUC-ROC and AUCPR. The findings of the experiments are summarised in Table 4. According to the results, the batch processing models (MCD, IForest, LSCP, and ELSCP) that were adapted to the streaming context utilising the reference window model outperformed the streaming models (RRCF and exact-STORM). In terms of AUC-ROC, both LSCP and ELSCP performed admirably, with ELSCP surpassing LSCP by 3.64 percent. MCD and IForest both yielded satisfactory results. Exact-STORM was at the baseline and did not appear to differentiate between the positive and negative classes. In terms of AUCPR, all models except Exact-STORM can distinguish between the positive and negative classes since their average AUCPR value is higher than the baseline of 0.087.