# On-Board Ship Detection in Micro-Nano Satellite Based on Deep Learning and COTS Component
## 5 Experimental Results
### Processing Performance  
This subsection tests the effectiveness of the proposed method. To quantitatively evaluate the performance of different frameworks, we use recall and precision as evaluation metrics of ship detection. We provide the definitions of precision and recall rate as  
\[Recall=\frac{Number\ of\ ship\ targets\ detected}{Number\ of\ ship\ targets\ in\ the\ dataset} \tag{3}\]  
\[Precision=\frac{Number\ of\ ship\ targets\ detected}{Number\ of\ detection\ boxes} \tag{4}\]  
Table 2 summarizes the experimental results of different methods. We also conducted ablation experiments on our dataset. The performance of S-HOG [11] is poor, because our dataset is complex and this unsupervised approach based on saliency and the S-HOG descriptor needs many empirical parameters. YOLOv3-tiny [44] is an improved version of YOLO, and it has the smallest model size but suffers the sacrifice of detection accuracy. Other methods in Table 2 are under our scheme. The method in [21] is not suitable for practical application due to the model size of 521.5 MB. Mobilenet [45] has a small model size, which is an efficient convolutional neural networks for mobile vision applications. Mobilenet uses depthwise separable convolution to reduce the parameters and the recall has a little decline at the same time. The decline of recall is unacceptable for on-board processing since the missing targets will not be transmitted to the ground receiver. The trade-off between performance and model size is always a difficult problem. For on-board ship detection, we think that the performance is more important than model size. Finally, we choose Resnet-50 as our baseline network. Resnet-50-1 and Resnet-50-4 were designed to evaluate the effect of the replacement of FC layers by GAP layer. The size of GAP layer depends on the output of the network and it is 7 \(\times\) 7 in Resnet-50-4. Resnet-50-4 has a performance improvement over Resnet-50-1 by a large margin and reduces the model size from 250.81 MB to 108.03 MB, which is very practical for on-board processing. The advantage of using \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Anchor** & \multirow{2}{*}{**Scale**} & **Aspect** & **Positive** & **Positive Samples** & **Average IoU of** \\
**Type** & & **Ratios** & **Samples/Image(s)** & **(IoU \textgreater{} 0.5)/Image(s)** & **Positive Samples** \\ \hline
**B-4** & 1.42, 3.48, 6.40, 10.50 & 1 & 13.60 & 7.32 & 0.579 \\
**B-12** & 1.42, 3.48, 6.40, 10.50 & 0.33, 1, 3 & 14.99 & 14.08 & 0.578 \\
**S-4** & 2.06, 5.29, 9.40, 15.19 & 1 & 23.83 & 22.88 & 0.604 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The statistics of different types of anchors. B and S represent original anchor and square anchor respectively, followed by the number of anchors. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Method** & **Anchor type** & **FC/GAP** & **Recall** & **Precision** & **Model Size (MB)** \\ \hline
**S-HOG** [11] & – & – & 0.623 & 0.587 & – \\
**YOLOv3-tiny**[44] & S-4 & FC & 0.807 & 0.771 & 30.1 \\
**Method in** [21] & S-4 & FC & 0.954 & 0.740 & 521.5 \\
**Mobilenet** [45] & S-4 & FC & 0.876 & 0.836 & 64.82 \\
**Resnet-50-1** & S-4 & FC & 0.920 & 0.774 & 250.81 \\
**Resnet-50-2** & B-4 & GAP & 0.933 & 0.707 & 108.03 \\
**Resnet-50-3** & B-12 & GAP & 0.959 & 0.757 & 108.12 \\
**Resnet-50-4** & S-4 & GAP & 0.979 & 0.816 & 108.03 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparisons of different methods. the strategy of square anchors can also be shown in Table 2. Resnet-50-2, Resnet-50-3, and Resnet-50-4 are designed to evaluate different anchor types. S-4 has a better performance than B-4 and B-12 since it can provide higher quality positive samples. This demonstrates that using square anchors has a good effect on the scheme. Overall, Resnet-50-4 gets the best performance with 97.9% recall and 81.6% precision. Table 3 is the performance comparisons by different NMS methods.