# Long-Tail Instance Segmentation Based on Memory Bank and Confidence Calibration
## 1 Introduction  
Object detection and instance segmentation are basic tasks in computer vision. Over the past decades, many studies of these tasks have been carried out from different perspectives. With the rapid development of neural networks, detection and segmentation tasks in computer vision have achieved unprecedented breakthroughs for common objects such as pedestrians or signs. Therefore, to obtain a computer vision model with better performance, the usual approach is to collect a large-scale dataset to train the model. There are always objective factors in producing the dataset that prevent the collection of the same number of samples for each category. For example, when creating an animal dataset, only one percent or even one-thousandth of the number of common animals such as cats and dogs can be collected due to the scarcity and harsh living conditions of the Northeast tiger. In this dataset, cats and dogs are the head category, the northeastern tigers are the tail category, and the presence of the tail category is very detrimental to the training of the model [1; 2; 3].Therefore, how to solve this problem has become a popular research topic in computer vision. Currently, effective methods include class-balanced resampling [4; 5], fine-tuning, meta-learning, causal reasoning [6], and few-shot learning techniques such as feature normalization. In addition, repeat factor sampling (RFS) [7] has become a standard simple resampling method. However, this method ignores the fact that an image often contains multiple objects of different categories, which results in the co-proliferation of regular classes and tail classes in the images. As a result, it cannot balance the number of classes in the training set. This directly causes the model to assign a larger confidence score to the head class in the testing phase, and it makes the classifier assign the head-class label to the segmented images more often. In this paper, to improve the resampling effect of small samples in long-tailed datasets, especially in instance segmentation, and to enhance the performance of long-tailed instance segmentation models, we propose a target-centric post-processable dynamic memory library. The main contributions are as follows:  
* First, the long-tail dataset is a kind of dataset with unbalanced samples, and a memory bank for rare classes is constructed to solve the sample unbalance problem. We use the forward-propagation region-of-interest (RoI) features to establish a dynamic memory bank that stores them and the target frame coordinates. The characteristic information of the tail classes is stored and used in a targeted manner. This enables efficient tail class resampling without further forward/backward propagation. * A post-process calibration is used to adjust each class logit, which changes the confidence scores between different classes for more accurate prediction by adjusting the logit of each class. The instance segmentation task also puts high demands on the quality of mask generation, so this paper seeks to improve not only the sample imbalance in the long-tailed dataset of instance segmentation, but also the quality of mask generation. To improve the quality of mask and reduce the computation, a mask method based on discrete cosine transform is used to improve the Mask R-CNN, which cannot capture the target edge details when generating a low-resolution mask and increases the training complexity when generating a high-resolution mask, to reduce the training complexity and improve the quality of mask generation at the same time. In summary, this paper proposes a target-centered post-processable dynamic memory bank model and uses the cosine transform method in the mask generation method. It solves the problem of small sample imbalance in the instance segmentation long-tail dataset, and the quality of mask generation for instance segmentation, so that the proposed method can be better applied to the application scenario of instance segmentation long-tail data.