# Benchmarking Deep Learning Models for Instance Segmentation
## 4 Review Method
### Speed-Focused Models
However, in Fast NMS, the already erased anchor box continues to be used for comparison to erase other anchor boxes. Fast NMS has a slightly worse performance because it removes more anchor boxes than NMS, but matrix operations can be performed using a GPU, resulting in significant speed improvements. **YolactEdge**[9] is a model based on YOLACT [8] that has two fundamental improvements at the system and algorithm levels so that it can perform inferences in real time on edge devices. The system was improved by using an NVIDIA TensorRT [22] to adjust the balance between speed and accuracy systematically while quantizing network parameters to 8-bit integer (int-8) or fp-16, which is the precision of a smaller bit range. A wider precision range requires more time for calculation and thus is more accurate. In addition, if the weights of the model components have different precision ranges, the balance between speed and accuracy depends on the components. Therefore, to maximize FPS without loss of accuracy, YolactEdge explored the optimal combination of int-8 and fp-16 weights for each component of the YOLACT by applying TensorRT. In particular, changing the prediction head to int-8 significantly decreases the segmentation accuracy. Hence, the authors converted all components except the predicted head and FPN to int-8, quadrupling the speed with the calibration step. Figure 10: YOLACT architecture. The mask result was graphically modified, not the actual inference result (COCO-val2017-00000021903 [10]). When YolactEdge was proposed, other real-time instance segmentation methods dealt with only static images. The authors who presented YolactEdge [9] argued that real-time applications are much more important in video applications, which require low latency and real-time inference. Therefore, the model was switched from static image processing to video processing, and training was performed using the YouTube Video Instance Segmentation dataset [28]. The improvement at the algorithm level involves using temporal redundancy when dealing with videos. In the keyframe case, backbone and pyramid features are computed, but in the non-keyframe case, only some of the backbone and pyramid features are computed and some of the pyramid features are transformed via _partial feature transform_ from the nearest previous keyframe in time (Figure 11). Through this process, the speed of YolactEdge is greatly accelerated because it avoids calculating the most expensive part of the backbone. To transform features quickly, object motion must be efficiently estimated, and thus FeatFlowNet, a modified network that follows the architecture of existing FlowNetS [29] (Figure 12a) but reuses the backbone's features and uses fewer convolution layers, was proposed (Figure 12b). In the feature transformation process, the flow map between the previous keyframe and current non-keyframe is estimated through FeatFlowNet; then, the features are transformed from the previous keyframe to the current non-keyframe through inverse warping, resulting in much faster calculations and identical performance. **Blendmask**[7] was proposed as an instance segmentation model that improves mask prediction based on an FCOS detector [26] with minimal modifications. The authors who presented Blendmask [7] argued that high-level features represent semantic information about instances more effectively than low-level features because the former have larger receptive fields than the latter, and that low-level features have the advantage of including more detailed information than high-level features without losing location information. Hence, the authors mixed the bottom-up and top-down approaches. Blendmask consists of two blocks: an FCOS detector that predicts the class and bbox of the instance; a mask branch that predicts the score maps and instance attentions, and then merges the scores with attentions (Figure 13). Figure 11: YolactEdge architecture.