# Probabilistic Ship Detection and Classification Using Deep Learning
## 3 Building a Sequence of Bounding Boxes  
In this section, we build a sequence for the bounding boxes using the boxes returned from the Faster R-CNN over time. In building the bounding box sequence, two practical issues had to be considered. The first issue was which bounding box to select at each time to create a reasonable sequence. The second issue involved how to handle the situation in which all the bounding boxes at time \(t\) did not make sense and when the target ship has apparently not been detected. To address these issues, we used the intersection over union (IoU) of the target bounding box and the predicted bounding boxes. Figure 2 illustrates two bounding boxes with IoU of 0.3, 0.4, and 0.9. For the two given bounding boxes \(B_{1}\) and \(B_{2}\), IoU computes the intersection of two boxes divided by the area of their union as follows:  
\[IoU(B_{1},B_{2})=\frac{area(B_{1}\cap B_{2})}{area(B_{1}\cup B_{2})} \tag{4}\]  
Concerning the first issue, we assumed that the target ships do not move rapidly at sea. Therefore, when the Faster R-CNN returns \(R\) bounding boxes from a given image in the \(t\)th frame, the bounding box with the largest IoU with the bounding box \(B^{t-1}\) in the previous frame is used as the bounding box \(B^{t}\) in the current frame:  
\[B^{t}=\underset{B^{t}_{r}\in\{B^{t}_{r},\ldots,B^{t}_{k}\}}{\operatorname{ argmax}}\ IoU(B^{t}_{r},B^{t-1}) \tag{5}\]  
where \(B^{t}_{r}\) denotes the \(r\)th predicted bounding boxes returned by Faster R-CNN in the \(t\)th frame.  
Concerning the second issue, when the detector in the current frame did not predict the position of the ship correctly and \(\underset{B^{t}_{r}\in\{B^{t}_{r},\ldots,B^{t}_{k}\}}{\operatorname{max}}\ IoU(B^{t}_{r},B^{t-1})< \epsilon_{thd}\), the target ship is likely to be missed. In this case, we enlarged \(B^{t}\) slightly from \(B^{t-1}\) by adding an offset to avoid missing ship detection, where \(\epsilon_{thd}\) denotes a threshold. This can be represented by:  
\[\begin{array}{ll}B^{t}&=B^{t-1}+\delta B\\ &=(v^{t-1}_{x},v^{t-1}_{y},v^{t-1}_{w},v^{t-1}_{h})+(\delta v_{x},\delta v_{y}, \delta v_{w},\delta v_{h})\\ &=(v^{t}_{x}-\delta v_{x},v^{t-1}_{y}+\delta v_{y},v^{t-1}_{w}+\delta v_{w},v^ {t-1}_{h}+\delta v_{h}),\end{array} \tag{6}\]  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \(k\) & **1** & **2** & **3** & **4** & **5** & **6** & **7** & **8** \\ \hline Label & Aircraft carrier & Destroyer & Submarine & Bulk carrier & Container ship & Cruise ship & Tugboat & Back-ground \\ \hline \end{tabular}
\end{table}
Table 1: Classes in the ship dataset.  
Figure 2: An illustration of two bounding boxes with an Intersection over Union (IoU) of (**a**) 0.3, (**b**) 0.5, and (**c**) 0.9.  
where \(\delta B=\left(\delta v_{x},\delta v_{y},\delta v_{w},\delta v_{h}\right)\) denotes an offset added to the bounding box. Figure 3 shows the update of the target ship-bounding box based on IoU.  
If \(R\) bounding boxes are predicted in the first frame, the initial bounding box was selected as the one with the highest detection confidence busing:  
\[B^{1}=\begin{array}{c}\operatorname*{argmax}\\ B^{1}_{r}\in\{B^{1}_{1},\dots,B^{1}_{R}\}\\ k=1,\cdots,C-1\end{array}p(\omega=k|B^{1}_{r}). \tag{7}\]  
For example, Faster R-CNN predicts four bounding boxes with class confidence in the first frame, as shown in Figure 4. Since the determined class of the predicted bounding box is the class with the highest confidence, the classes of the detected boxes from 1 to 4 are an aircraft carrier (0.34), bulk carrier (0.895), bulk carrier (0.668), and a destroyer (0.422), respectively. In this case, from Equation (7), we selected bounding box 2 as the initial bounding box in the video.  
Figure 3: IoU tracking. (**a**) IoU is equal to or larger than the threshold and (**b**) IoU is less than the threshold. If IoU is less than the threshold, the bounding box in frame \(t-1\) increases slightly and is updated as the target ship-bounding box to avoid missing ship detection.