# Benchmarking Deep Learning Models for Instance Segmentation
## 5 Experiments
### Quantitative Results
#### 5.1.2 Inference Speed  
As shown in Table 2, the speed of the accuracy-focused models in the RTX 3090 environment was 1.9 FPS for Dual-Swin-L and 8.1 FPS for SOTA, both of which are slower than the 23.2 FPS result of Mask R-CNN. Additionally, in the GTX 1080 Ti environment, the speed was 0.8 FPS for Dual-win-L and 4.1 FPS for SOTA, whereas the speed of Mask R-CNN was estimated to be 11.7 FPS. The speeds of the speed-focused models in RTX 3090 environment were 53.5 FPS for YolactEdge, 52.2 FPS for BlendMask, 45.2 FPS for CenterMask, 41 FPS for YOLACT, and 37 FPS for SOLOv2, all of which were faster than the 23.2 FPS of Mask R-CNN. YolactEdge was estimated to run at 177.3 FPS when TensorRT optimization was applied. Additionally, in the GTX 1080 Ti environment, the speeds were 37.8 FPS for BlendMask, 32.6 FPS for YolactEdge, 30.8 FPS for CenterMask, 26.6 FPS for SOLOv2, and 21.5 FPS for YOLACT, all of which were faster than the 11.7 FPS of Mask R-CNN. YolactEdge was estimated to run at 81.6 FPS when TensorRT optimization was applied. \begin{table}
\begin{tabular}{l l l l l l l l l l} \hline \hline
**Method** & **Model** & **Reference GPU** & **Tested GPU** & **Tested FPS** & **AP\({}_{mask}\)** & **AP\({}_{base}\)** & **AP\({}_{S}\)** & **AP\({}_{M}\)** & **AP\({}_{L}\)** \\ \hline \multirow{2}{*}{Dual-Swin-L} & \multirow{2}{*}{DB-Swin-L [32]} & RTX 3090 & 1.9 & \multirow{2}{*}{51.0} & \multirow{2}{*}{59.1} & \multirow{2}{*}{32.2} & \multirow{2}{*}{54.3} & \multirow{2}{*}{69.0} \\ \cline{5-8}  & & & GTX 1080 Ti & & & & & & \\ \cline{5-8}  & & & RTX 3090 & 8.1 & & & & & \\ \cline{5-8}  & & & GTX 1080 Ti & 4.1 & & & & & \\ \hline \multirow{2}{*}{Mask R-CNN} & \multirow{2}{*}{R101-FPN [34]} & NVIDIA V100 & RTX 3090 & 23.2 & \multirow{2}{*}{38.6} & \multirow{2}{*}{42.9} & \multirow{2}{*}{19.4} & \multirow{2}{*}{41.3} & \multirow{2}{*}{55.3} \\ \cline{5-8}  & & (17.8 FPS) [34] & & GTX 1080 Ti & 11.7 & & & & \\ \hline \multirow{2}{*}{Centermask} & \multirow{2}{*}{\begin{tabular}{l} centermask2-lite-V-39-eSE- \\ FPN-ms-dx [35] \\ \end{tabular} } & NVIDIA Titan Xp & RTX 3090 & 45.2 & \multirow{2}{*}{36.7} & \multirow{2}{*}{40.9} & \multirow{2}{*}{17.2} & \multirow{2}{*}{39.8} & \multirow{2}{*}{53.4} \\ \cline{5-8}  & & (35.7 FPS) [35] & & GTX 1080 Ti & 30.8 & & & \\ \hline \multirow{2}{*}{SOLOv2} & SOLOv2\_Light\_512\_DCN\_ & NVIDIA V100 & RTX 3090 & 37.0 & \multirow{2}{*}{36.4} & \multirow{2}{*}{-} & \multirow{2}{*}{13.1} & \multirow{2}{*}{40.1} & \multirow{2}{*}{57.8} \\ \cline{5-8}  & R50\_3x [36] & & (29.4 FPS) [36] & & GTX 1080 Ti & 26.6 & & & \\ \hline \multirow{2}{*}{Blendmask} & \multirow{2}{*}{DLA,34\_4x [37]} & GTX 1080 Ti & RTX 3090 & 52.2 & \multirow{2}{*}{36.3} & \multirow{2}{*}{40.8} & \multirow{2}{*}{16.0} & \multirow{2}{*}{39.6} & \multirow{2}{*}{53.9} \\ \cline{5-8}  & & (32.1 FPS) [37] & & GTX 1080 Ti & 37.8 & & & & \\ \hline \multirow{2}{*}{YOLACT} & \multirow{2}{*}{\
\begin{tabular}{l} val2-im700\_54\_800000 [38] \\ \end{tabular} } & NVIDIA Titan Xp [8] & RTX 3090 & 41.0 & \multirow{2}{*}{30.9} & \multirow{2}{*}{33.4} & \multirow{2}{*}{12.0} & \multirow{2}{*}{34.3} & \multirow{2}{*}{48.9} \\ \cline{5-8}  & & GTX 1080 Ti & 21.5 & & & & & \\ \hline \multirow{2}{*}{YolactEdge} & \multirow{2}{*}{
\begin{tabular}{l} val2-edge\_54\_800000 [59] \\ \end{tabular} } & RTX 2080 Ti [9] & RTX 2080 Ti [9] & RTX 2080 Ti [9] & 29.8/29.9 * & 32.5/32.4 * & 10.1/9.9 * & 32.3/31.9 * & 50.2/50.4 * \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Benchmark results on MS COCO val2017 (: when impossible to measure due to network structure; “*” indicates the FPS results with NVIDIA TensorRT optimization [22]). Figure 16: AP and FPS results for various instance segmentation methods on COCO-val2017: (**a**) RTX 3090 vs GTX 1080 Ti AP\({}_{mask}\) results for FPS; (**b**) AP\({}_{mask}\) and AP\({}_{S}\) results on the RTX 3090 and GTX 1080 Ti; (**c**) AP\({}_{mask}\), AP\({}_{M}\) results on the RTX 3090 and GTX 1080 Ti; (**d**) AP\({}_{mask}\), AP\({}_{L}\) results on the RTX 3090 and GTX 1080 Ti.