# Benchmarking Deep Learning Models for Instance Segmentation
## 6 Discussion  
We categorized the experimental methods as accuracy-focused and speed-focused models, presented an overview of each model, and summarized their contribution. In addition, we ran an inference with MS COCO val2017 dataset [10] in a fixed test environment based on a GPU (RTX 3090 or GTX 1080 Ti) and compared the AP and FPS results of each model. As shown in Table 1 and Figure 16, the accuracy-focused models obtained relatively higher AP scores and relatively lower inference speed than the speed-focused models. Dual-Swin-L and SOR outperformed Mask R-CNN in terms of AP. Furthermore, Dual-Swin-L has a noticeably higher AP than other models. We can infer from these results that the Transformer structure is an effective approach not only in the NLP field but also in computer vision. Although speed-focused models obtained lower AP scores than Mask R-CNN, all models run at real-time speed in the RTX 3090. Blendmask, Centermask, and YolactEdge also run at real-time speed in the GTX 1080 Ti, however, SOLOv2 and YOLACT do not run in real-time in the GTX 1080 Ti. In addition, we can confirm that AP and FPS have a trade-off relationship in Figure 17. Therefore, it seems that an appropriate trade-off between speed and accuracy should be considered for real-time use. As stated above, speed-focused models have a faster inference speed than accuracy-focused models. Speed-focused models using the FPN structure have a speed of over 30 FPS in RTX 3090. Conversely, Dual-Swin-L [2] and SORR [3], which use the Transformer structure, have a speed below 10 FPS. Mask R-CNN [4] was categorized as an accuracy-focused model, but because it uses an FPN structure, it has a relatively fast speed of 23.2 FPS in the RTX 3090 environment. Although they were not tested in this study,  
Figure 21: Visualization of instance segmentation results (COCO-val2017-000000329323 [10]). (**a**) Dual-Swin-L; (**b**) SORR; (**c**) Mask R-CNN; (**d**) BlendMask; (**e**) CenterMask; (**f**) SOLOv2; (**g**) YOLACT; (**h**) YolactEdge. SipMask [52] and OrienMask [53], which can be categorized as speed-focused models, also used an FPN structure. OrienMask [53], which uses a Darknet-53 backbone, has reportedly achieved an AP of 34.5 and speed of 41.9 FPS on the COCO val2017 dataset in an RTX2080 Ti environment. SipMask [52], which uses a ResNet101-FPN backbone, has reportedly achieved an AP of 32.8 and speed of 31.2 FPS on the COCO test-dev dataset in a Titan Xp environment. Therefore, from the results on network structures released thus far, an FPN can be considered when designing a real-time model. As shown in Table 2, when TensorRT optimization [22] was used for YolactEdge [9], the speed increased 2.5 times or more in the GTX 1080 Ti environment and 3.3 times or more in the RTX 3090 environment. However, no significant differences were observed in AP results. Therefore, we can expect a meaningful speed improvement by applying TensorRT to processing units that support CUDA. Furthermore, the recently unveiled NVIDIA Jetson AGX Orin [54] reportedly performed up to eight times faster than NVIDIA Jetson AGX Xavier [55]. Given that YOLACT achieved a 27.3 FPS in the NVIDIA Jetson AGX Xavier [39], high complexity models will be able to meet real-time requirements through TensorRT optimization in the NVIDIA Jetson product line, including the Jetson AGX Orin. As shown in Table 1, different GPU environments lead to a difference in the inference speed but yield the same AP scores. Table 3 shows that comparison of the speeds of the methods in an RTX 3090 environment relative to that in a GTX 1080 Ti environment confirms that the improvements in speeds differ. Figure 18 also compares the GFLOPS and speed of each method. From these experimental results, we can confirm that the speed is proportional to the GFLOPS; however, the increase is not as significant. [PERSON] et al. [11] presented the performance of mobile SoCs in the fp-16 network and compared the performance of Intel CPUs and NVIDIA GPUs in the fp-32 network.