# CenterInst: Center-Based Real-Time Instance Segmentation
## 1 Introduction  
Image segmentation addresses the problem of labeling pixels. There are different types of segmentation tasks, such as semantic, instance, and panoptic segmentation. Instance segmentation can be described as a combination of image segmentation and object detection. It aims to detect all objects and label all pixels that belong to the same object. Instance segmentation is often time-consuming. For SOLO [1, 2], researchers have started to pay attention to real-time instance segmentation. However, SOLO is based on dense detection. Therefore, nonmaximum suppression (NMS) is utilized for postprocessing, which limits the scale of the model. SparseInst [3] uses sparse detection to improve SOLO by eliminating NMS and completing the prediction of instance segmentation by generating an instance activation map. Instead of a convolution-based decoder, FastInst [4] uses Transformer to construct a double-channel decoder to complete real-time instance segmentation. Although FastInst achieves better performance, there are still some problems in real-time instance segmentation: (1) The many-to-one problem in sparse detection occurs easily. FastInst takes advantage of a semantic segmentation network to extract proposal queries. However, semantic segmentation-based methods cannot easily extract a small number of effective proposal queries for one instance. (2) Feature sampling is not efficient. FastInst uses the cross-attention of Transformer to complete feature sampling from an E3 feature map. It relies on single-scale sampling in the E3 layer, which results in limited performance. To improve accuracy, the segmentation results of the previous stage are regarded as a mask to help the decoder focus less on irrelevant areas. However, it is not efficient enough, because a calculation of the similarity of all pixel features is still needed. Therefore, the multiscale feature cannot be used due to the low speed resulting from sampling the whole image. Additionally, the irrelevant features from the whole image lower the performance. (3) The pixel features are updated too many times. FastInst uses the matching results of the first iteration as a limitation, and the training method introduced by GT mask-guided training is not effective enough. The pixel features need to update at every stage. In this study, we solved the above problems by developing CenterInst, a novel center-based real-time instance segmentation model following the meta-architecture of FastInst. CenterInst boasts lower computational complexity, faster computation speed, and higher inference accuracy than FastInst. Our contributions are summarized below:  
We developed an innovative center-based instance segmentation architecture that leverages the central information contained within instances. In this architecture, we generate initial queries guided by the center and perform local sampling based on center information. We then update query vectors using the sampled features. Finally, we generate mask kernels based on the updated query vectors to predict the instance segmentation results. Inspired by CenterNet [5], we propose a center-guided query selector, which achieves more accurate initialization query selection by predicting the center point probability of each pixel on the feature map. Compared to segmentation-based methods, this approach alleviates the issue of generating too many queries for the same object. We propose a center-guided sampling-based query decoder, which achieves local sampling based on instance centers. Compared to global sampling, it significantly reduces the number of sampled features and improves the correlation between sampled features and queries, thereby enhancing accuracy while maintaining inference speed. To further improve the model's inference speed and optimize its real-time performance, we developed a lightweight inference structure and a corresponding training method. This structure enhances the inference speed by reducing the frequency of pixel feature updates.