# Smart Black Box 2.0: Efficient High-Bandwidth Driving Data Collection Based on Video Anomalies
## 4 Materials and Methods
### Updated SBB Design  
The updated SBB is separated into four processes running in parallel: video capture, buffer management, value estimation, and prioritization. Figure 1 describes the updated \begin{table}
\begin{tabular}{l l l} \hline \hline
**Name** & **ID** & **Description** \\ \hline N & 0 & No anomaly \\ ST & 1 & Collision with another vehicle which starts, stops, or is stationary \\ AH & 2 & Collision with another vehicle moving ahead or waiting \\ LA & 3 & Collision with another vehicle moving laterally in the same direction \\ OC & 4 & Collision with another incoming vehicle \\ TC & 5 & Collision with another vehicle that turns into or crosses a road \\ VP & 6 & Collision between vehicle and pedestrian \\ VO & 7 & Collision with an obstacle in the roadway \\ OO & 8 & Out-of-control and leaving the roadway to the left or right \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Event Classes in the DoTA dataset [15]. SBB pipeline. Our code is available at [[https://github.com/rzf16/sbb2_algs](https://github.com/rzf16/sbb2_algs)]([https://github.com/rzf16/sbb2_algs](https://github.com/rzf16/sbb2_algs)) (accessed on 8 February 2021). **Video Capture** reads video input and publishes each video frame to value estimation and buffer management. This module remains unchanged from the original SBB. **Value Estimation** assigns a value \(v_{t}\in[0,1]\) for each frame to be used in buffer management and storage prioritization. The value estimation module first executes object detection, object tracking [5], and optical flow estimation [35]. The outputs are then used in video anomaly detection and online action detection, which are used to compute the value. Details on this calculation can be found in Section 4.3. The value estimation method used differs significantly from [13]. In [13], perfect EOI detection using pre-defined rules was assumed, and the data value was computed based on detected EOIs. In this paper, we instead use video anomaly detection and action detection methods for generalized EOI detection and calculate the data value based on their output scores. More details on the value estimation of the original SBB can be found in Section 3.2. More details on our updated value estimation can be found in Section 4.3. **Buffer Management** groups frames into buffers using the DMM from [13] after receiving each frame from Video Capture and its corresponding value from Value Estimation. The similarity of a data frame to the current buffer is computed as the percentage of object tracking IDs in the frame that have already appeared in the buffer. With \(A\) being the set of tracking IDs in the frame and \(B\) being the set of tracking IDs that have appeared in the buffer, we compute the similarity \(\xi_{t}=\frac{|A\cap B|}{|A|}\). Once the DMM terminates, LBO solves an optimization problem over the output buffer to determine the compression quality of each frame. According to [13], a decoupled LBO strategy can optimize the compression quality of a single frame independently of all other frames in the buffer. Given constant \(\frac{\eta}{\xi}\), the uncoupled LBO objective function for frame \(f_{t}\) is:  
\[ \begin{split}&\min_{d_{t}}c_{t}\phi(d_{t})-\frac{\eta}{\xi} \vartheta_{t}d_{t}\\ &\text{subject to }d_{t}\in[0,1]\end{split}  \tag{5}\]  
Figure 1: The updated smart black box (SBB) data recording pipeline. Green blocks denote new modules that were not present in the original SBB [13]. Black arrows indicate logic flow, and blue arrows indicate data flow. LBO, local buffer optimization. where \(\eta,\zeta\geq 0\) are weighting parameters and \(\phi(d_{t})\) maps from the compression quality to the compression ratio. Note that \(\phi(d_{t})\) increases monotonically over \(d_{t}\in[0,1]\). In this paper, we use the \(\phi\) function of JPEG compression on real-world driving data following [12]. Throughout the paper, the values \(\eta=0.9\) and \(\eta=1.7\) are used based on [13]. These parameters were assigned to maximize the value-per-memory (VPM) of the recorded data.