## 5 Experimental Results and Analysis
### Comparison Overview  
In this section, we compare the proposed method with a vehicular motion-based intrusion detection system named MIDS [7] and a replicator neural network-based vehicle anomaly detection system named RNet [8] to further evaluate the capability of our proposed anomaly detection model. Specifically, we first implement the detection methods under mixed anomaly scenarios with fixed anomaly samples (150 anomaly samples), i.e., the anomaly duration, to investigate their overall detection performance. Then, we set anomaly signals with different numbers of anomaly durations (50, 100, 150, and 200 anomaly samples) to evaluate the performance under different anomaly extents. The results in Table 10 shows that compared with the other methods, our CWT-CNN model has a better detection performance. Figure 6 shows the graphical depiction of the detailed comparison between the proposed model and other methods. It can be seen that our proposed model outperforms the compared methods under all evaluation criteria. This is partially because the MIDS method only extracts the time domain information, i.e., the signals correlation; there are always some unseen patterns that the method cannot capture and thus, it underperforms compared with our model. In addition, though MIDS uses CUSUM as a detector module to improve the detection accuracy, it requires setting an appropriate update threshold, sensitivity threshold, and detection threshold, which also brings hand-crafted uncertainty and bias and increases the difficulties of model designing. Considering the RNet method, inputting the raw in-vehicle sensor signals in the neural network will decrease the feature learning capability, as the signals have plentiful noise and different frequency components. Our proposed model uses the CWTS to obtain the complete time and frequency domain information, and the CNN model enables the method to take full advantage of its powerful learning ability to extract more abstract features from each element in the CWTS for a better anomaly detection performance. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Acc** & **Sens** & **Prec** & **F1** \\ \hline
96.28 & 95.30 & 97.21 & 96.24 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Detection performance of the proposed model under mixed anomaly scenarios. \begin{table}
\begin{tabular}{c c} \hline \hline
**Module** & **Detection Time (ms)** \\ \hline CWT & 6.08 \\ CNN & 1.87 \\ CWT-CNN & 7.98 \\ Timeliness & \textless{}10 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Detection time of the proposed model. Table 11 shows the comparison results of different anomaly durations. For each method, we first intercept the real-time in-vehicle sensor data into each detection window by using sliding window strategy. Then, in each detection window, we set anomaly sample numbers of 50, 150, 200, and 250 to evaluate the detection performance under different anomaly severities. It is seen that the detection performance of MIDS considerably drops as the anomaly duration decreases; it reaches the worst performance, with an 83.58% accuracy at the shortest duration, i.e., row 1. This is because the MIDS method requires a large number of anomaly sample accumulation in a detection window to result in a severe variation in the detector indicator to conduct anomaly detection. Thus, in the case of a shorter anomaly duration, it will not cause large magnitude changes within the detector, so the performance will be relatively poor. Because RNet utilizes changes in the reconstructed signal value to detect, though its performance slightly weakens as the anomaly duration decreases, its overall detection performance is worse than our proposed model. As is seen from the result of our proposed model, our method perform reasonably better compared with the other methods under both short and long durations. As our method detects anomalies by discovering deviations, any deviations caused by anomaly behaviors at the early stage will be detected.