# Smart Black Box 2.0: Efficient High-Bandwidth Driving Data Collection Based on Video Anomalies
## 5 Experiments
### Results  
Experimental results for the SBB are presented and discussed below. In general, a high storage size and decisions are desirable for anomalous data, while the opposite is true for normal data. **SBB data compression:** SBB data compression statistics with no memory limit are presented in Table 5. It can be seen that the storage cost of normal frames is significantly reduced (427.20 GB to 65.53 GB, 85%) by the SBB. This leads to a 24.4% increase in the ratio of anomalous data storage-to-normal data storage. Both the average (**avg. **) and median (**med. **) compression factor decisions of the SBB are higher for the anomalous frames, indicating that the SBB is able to identify and preserve anomalous frames over normal ones. Figure 2 displays normal frames that were highly compressed by the SBB along with preserved anomalous frames; Figure 3 shows two failure cases where anomalous frames were mistakenly compressed. Both of these failures showcase a lack of robustness against cases where anomalous objects are occluded. However, the decision difference between normal and anomaly frames is not as significant compared to the simulation experiment in [13] due to the fact that the anomalous event detection in the simulation was 100% accurate, while VAD on real-world data is far from perfect. Moreover, the median decision for a normal frame is significantly lower than the mean, indicating that there are outlier normal frames with unusually high value scores. The standard deviation (**std. **) of anomalous frames is significantly larger than that in the simulation experiment (0.26 vs. \(\sim\)0.02), showing how inaccurate VAD and OAD reduces the SBB's efficiency on real-world data. Figure 2: Compressed normal frames (**left**) and preserved anomalies (**right**). The limitations of VAD and OAD are further shown by evaluating the performance of the SBB given ground-truth labels as VAD and OAD scores. The anomalous-to-normal memory ratio increases by 1610%, driven by the massive difference in decisions between normal and anomalous frames. This upper-bound performance of the SBB indicates that as anomaly detection techniques continue to improve, the performance of the SBB will improve as well. Priority queue vs. FIFO:Table 6 compares the recorded frames of a prioritized recording system against that of an FIFO queue at memory limits of \(M=3.125\) GB, 6.25 GB, 12.5 GB, and 25 GB. These values represent a non-trivial amount of data to upload (depending on Internet connection quality) assuming continuous Internet access is not available. In all scenarios, the prioritized recording saved fewer normal frames and more anomalous frames than with the FIFO strategy. We also note that while the anomaly ratio stays roughly the same in each memory limit for the FIFO queue, the ratio increases at each level for the priority queue. The prioritization strategy of the SBB removes \(\sim\)95% of the normal frames while still recording \(\sim\)10% anomalous frames at \(M=3.125\) GB. Compared to the FIFO queue, the anomalous-to-normal count ratio of SBB-recorded data is \(\sim\)25% to \(\sim\)100% higher. \begin{table}
\begin{tabular}{l l l l l} \hline \hline  & & **Normal** & **Anomaly** & **Anomaly Ratio** \\ \hline \multirow{2}{*}{Raw Data} & \# of Frames & 3,967,977 & 16,768 & \multirow{2}{*}{0.41\%} \\  & size (GB) & 427.20 & 1.76 & \\ \hline \multirow{2}{*}{SBB w/VAD+OAD} & size (GB) & 65.53 & 0.33 & 0.51\% \\  & **avg.**\(d_{i}\) & 0.51 & 0.58 & \multirow{2}{*}{6.60\%} \\  & **med.**\(d_{i}\) & 0.55 & 0.63 & \\  & **std.**\(d_{i}\) & 0.24 & 0.26 & \\ \hline \multirow{2}{*}{SBB w/GT VAD+OAD} & size (GB) & 11.05 & 0.73 & 6.60\% \\  & **avg.**\(d_{i}\) & 0.00 & 0.92 & \\  & **med.**\(d_{i}\) & 0.00 & 0.92 & \\  & **std.**\(d_{i}\) & 0.03 & 0.00 & \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Raw, SBB-compressed and ground-truth (GT) SBB-compressed data statistics on the BDD100K+DoTA dataset. VAD, video anomaly detection; OAD, online action detection. Figure 3: Compressed anomaly failure cases. * indicates non-ego.