# Dynamic Detection and Recognition of Objects Based on Sequential RGB Images
## 4 Experiments and Analysis
### Results of LSMNet  
In this experiment, we used a fully convolutional UNet with skipping connection as the basic architecture of LSMNet. The learning rate was set as 0.0002, the batch size was 6, and the weights were \(\omega_{1}=\omega_{2}=\omega_{3}=100\). Cropping, rotation, channel fusion, and color jitter were used to augment the dataset. The curves of the training loss are illustrated in Figure 9. The \(l_{LSM}\) curves are above that of \(l_{mask}\), as LSMs are more difficult for the pix2 pix network to learn than binary masks. The fluctuations of \(l_{LSM}\), \(l_{mask}\), and \(l_{c}\) are almost synchronous, and it means that the consistent constraint is violated more seriously when the predicted LSM and mask are poor. The fluctuations would be eliminated if a bigger training set is accessible. Because the objects in the test set were new for the model, the loss of the test set was unsurprisingly much larger than that of the training set. In Figure 10, a few samples of the predicted LSMs on the training set are shown. LSM output suffers from small noisy regions (Samples 1, 3, and 5), which are filtered in the consistency outputs. However, some tiny objects are missed. The performance of LSMNet on small objects would be discussed further in the experiment on the COCO-car dataset. We compared LSMNet with YOLACT [14] to illustrate its performance. It should be noted that we fine-tuned the model of YOLACT, which is pretrained on COCO2017 [18]. The results of the videos are listed in Table 4. Because YOLACT is developed based on a one-stage object detection framework, the segmentation results are affected by the detection task. Moreover, YOLACT is apt to recall those objects ever seen in COCO2017 due to that the OCIS dataset is much smaller than COCO2017. As shown in Figure 11, some objects with large volumes are missed by YOLACT. LSMNet performs very well if there is no crowding. However, when there exists adjoin or occlusion among objects, it is hard to determine the threshold for extracting the peaks of LSM. Some poor results of LSMNet are presented in Figures 12 and 13. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{2}{c}{**Level 1**} & \multicolumn{2}{c}{**Level 2**} & \multicolumn{2}{c}{**Level 3**} \\ \cline{2-7}  & TPR & FPR & TPR & FPR & TPR & FPR \\ \hline YOLACT & 0.7532 & 0.0987 & 0.7398 & 0.1035 & 0.7794 & 0.2335 \\ LSMNet & 0.9437 & 0.0224 & 0.7098 & 0.1362 & 0.6277 & 0.3361 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: OCIS results on videos. Figure 9: Training loss curves of LSMNet on the OCIS dataset. Figure 11: Poor results of YOLACCT on videos. Figure 10: Results of LSMNet on the OSIC dataset. (**a–f**) Original image, mask ground truth, LSM ground truth, mask output, LSM output, and consistency output, respectively. To analyze the performance of LSMNet, we compared it with SCNet [41] and YOLACT on the COCO-car dataset. They are trained with two NVIDIA RTX2080 TI GPUs based on the open framework mmdetection [42], which is developed by SenseTime. The results on COCO-car are listed in Table 5. AP is the mean AP@IoU = 0.50:0.95, AP\({}_{L}\), AP\({}_{M}\), and AP\({}_{S}\) are the mean AP@IoU on cars of the large, medium, and small size, respectively, and fps is observed on an NVIDIA GTX1660 TI CPU. SCNet performed much better than the other two methods due to its introduction of the pregenerated stuffthingmaps. However, its cascade architecture made it much slower than the other two methods. LSMNet performed close to YOLACT on large and medium cars, but much poorer on small objects. The reason is that LSMNet extracts pixel-level information first and aggregates it to global semantic information. Noisy regions in the predicted LSM disturbed the segmentation of small objects. As the process of finding contours cannot be realized in the form of a differentiable module, the label information of the bounding boxes cannot be backpropagated to the pix2 pix architecture.