SOLQ: Segmenting Objects by Learning Queries
Year: 2021

Abstract:
In this paper, we propose an end-to-end framework for instance segmentation. Based on the recently introduced DETR [1], our method, termed SOLQ, segments objects by learning unified queries. In SOLQ, each query represents one object and has multiple representations: class, location and mask. The object queries learned perform classification, box regression and mask encoding simultaneously in an unified vector form. During training phase, the mask vectors encoded are supervised by the compression coding of raw spatial masks. In inference time, mask vectors produced can be directly transformed to spatial masks by the inverse process of compression coding. Experimental results show that SOLQ can achieve state-of-the-art performance, surpassing most of existing approaches. Moreover, the joint learning of unified query representation can greatly improve the detection performance of DETR. We hope our SOLQ can serve as a strong baseline for the Transformer-based instance segmentation. Code is available at https://github. com/megvii-research/SOLQ.

text: In this paper, we propose an end-to-end framework for instance segmentation. Based on the recently introduced DETR [1], our method, termed SOLQ, segments objects by learning unified queries. In SOLQ, each query represents one object and has multiple representations: class, location and mask. The object queries learned perform classification, box regression and mask encoding simultaneously in an unified vector form. During training phase, the mask vectors encoded are supervised by the compression coding of raw spatial masks. In inference time, mask vectors produced can be directly transformed to spatial masks by the inverse process of compression coding. Experimental results show that SOLQ can achieve state-of-the-art performance, surpassing most of existing approaches. Moreover, the joint learning of unified query representation can greatly improve the detection performance of DETR. We hope our SOLQ can serve as a strong baseline for the Transformer-based instance segmentation. Code is available at https://github. com/megvii-research/SOLQ.

heading: 1 Introduction
text: Instance segmentation, serving as one of visual detection tasks, not only locates instances of different categories but also generates pixel-level mask for each instance. State-of-the-art instance segmentation methods [2, 3, 4, 5] follow the two-stage paradigm, which first performs object detection and then segments the masks within detected boxes by RoIAlign [2]. Those methods are relatively easy to be optimized thanks to the deployment of mature object detectors [6, 7]. However, the segmentation branch heavily relies on the detection branch, making it hard to achieve better joint learning of multiple tasks. Some recent works [8, 9, 10, 11] build instance segmentation frameworks on top of anchorfree object detectors [12, 13] to remove the ROI-Cropping operation, reducing the effect of feature misalignment. For example, CondInst [9] based on FCOS [12] employs dynamic convolutions [14] to perform instance segmentation. YOLACT [8] models the instance segmentation as a combination of prototypes weighted by learned mask coefficients for each anchor. However, the weights of dynamic convolutions or the mask coefficients are still generated by instance proposals.
Regardless of the bounding boxes, SOLO [15] introduces the notion of "instance categories" and segments objects by locations. SOLOv2 [16] further solves the problem of inefficient mask representation by introducing dynamic convolution and so-called matrix Non-Maximum Suppression (NMS). Though SOLO directly outputs instance masks based on locations, hand-crafted post-processes, like NMS, are still required to remove duplicated predictions. Also, the performance of small objects is far from satisfactory due to the imbalance of location samples between the large and small objects. Building an end-to-end instance segmentation framework is still a remaining problem.
Our work is inspired by DETR [1], which first proposes the end-to-end solution for object detection. In DETR, object detection is regarded as a set prediction problem and objects are represented by
∗Equal contribution. This work is supported by The National Key Research and Development Program of China (No.2017YFA0700800) and Beijing Academy of Artificial Intelligence (BAAI).
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
learnable query embeddings. How to encode the spatial binary mask into such an end-to-end system is an opening question. As shown in Fig. 1(a), DETR is further extended to panoptic segmentation by directly reshaping the learnable embeddings into spatial domain and building a FPN-style [17] network to produce the final mask predictions. However, both the Transformer encoder and decoder fail to model the spatial information well. Therefore, it is inappropriate to generate the spatial mask based on such query embeddings. Besides, the spatial mask labels used for supervision are of large resolutions, which results in high computation cost and makes it separated to learn the detection and mask branches2. So we need to find one mask representation that satisfies the following conditions: 1) The representation can naturally convert object mask from spatial domain to embedding domain; 2) The process that encodes the spatial masks into embeddings should be reversible; 3) Mask embeddings encoded can keep the principle components of spatial mask. We turn to methods in literature for help and surprisingly find that classical compression coding methods (e.g. Sparse Coding [18]) just satisfy these conditions mentioned above. The mask embeddings can be simply generated from the learnable queries. In training phase, ground-truth spatial mask of each instance can be projected into low-dimensional mask embedding by compression coding and the mask embeddings are used to supervise the learning of predicted mask embeddings. In inference phase, binary spatial mask can be reconstructed from predicted mask embedding by the inverse process of compression coding.
With these analysis, we explore how to better encode the spatial mask into the end-to-end object detectors in this paper. Based on DETR, our proposed method, termed SOLQ, segments objects by learning queries. In SOLQ, we formulate the instance segmentation as the joint learning of unified query representation (UQR). The UQR learned can be used to perform parallel predictions for three sub-tasks (classification, localization and segmentation) simultaneously and all predictions are obtained in a regression manner (see Fig. 1(b)). In this way, SOLQ directly outputs instance masks together with corresponding class confidences and box coordinates. The learning of UQR can be divided into two parts: generating instance-aware query embeddings and joint supervision of multi-task learning. Specifically, all candidate instances are initialized with several learnable queries, which interact with extracted image features in Transformer decoder to produce the instance-aware query embeddings. The instance-aware query embeddings are further input to three branches of sub-tasks, which contains several linear projection layers, to generate three sub-task vectors. For the classification and regression branches, we follow the same supervisions as in DETR [1]. For the mask branch, we conduct implicit supervision with the help of mask compression coding mentioned above.
To summarize, our contributions are:
• We propose an end-to-end framework for instance segmentation based on DETR. SOLQ formulates the instance segmentation as the joint learning of UQR. In UQR, the mask representation can be converted from spatial domain into embedding domain, which is consistent with the learnable query embeddings in DETR.
• Experiments show that SOLQ with ResNet101 achieves 40.9% mask AP and 48.7% box AP on the challenging MS COCO dataset [19] without bells and whistles, outperforming SOLOv2 by 1.2% mask AP and 6.1% box AP. It is worthy noting that SOLQ can improve 2.0% in box AP compared to DETR thanks to the joint learning of UQR.

heading: 2 Related Work
text: Instance Segmentation Instance segmentation is a classic but challenging computer vision task. It is required to output each object instance in image with instance-level category label and localization, along with pixel-level mask simultaneously. Currently, there are mainly three categories of instance segmentation methods: top-down, bottom-up and directly-predict methods. Top-down approaches [20, 2, 8, 21, 22, 23, 3, 5, 24] follow the detect-then-segment pipeline. They first generate bounding boxes by object detectors and segment the masks by ROIAlign [2] or dynamic convolutions [9]. Bottom-up methods [25, 26, 27, 28] learn per-pixel embeddings via semantic label and then cluster them into instance groups. For directly-predict methods, PolarMask [29] employs polar coordinates to represent mask contours. Latest SOLO [15] and SOLOv2 [16] directly segment the objects by locations without dependence on bounding boxes or embedding learning. QueryInst [24] and ISTR [30] extend the Sparse RCNN [31] to perform end-to-end instance segmentation. In this paper, we
2Train the object detector first and then freeze the weights of object detector to train segmentation branch.
explore an end-to-end instance segmentation solution by learning an unified query representation without any post-processing procedures, like Non-Maximal Suppression (NMS).
Transformer in Vision Transformer [32] introduces the self-attention mechanism to model longrange dependencies, and has been widely applied in natural language processing (NLP). Recently, several works attempted to involve the Transformer architecture into various computer vision tasks and showed promising performances. The non-local block [33] is first proposed to enhance video recognition by aggregating spatial information. After that, CCNet [34] further extends the selfattention via sparse attention in semantic segmentation. DETR [1] and Deformable DETR [35] adopt learnable queries and Transformer architecture together with bipartite matching to perform object detection in end-to-end fashion, without any hand-crafted process such as NMS. IPT [36] proposes a transformer-based pretrained network for low-level image processing. ViT series [37, 38, 39, 40] take an image as a sequence of patches and achieve the cross-patch interactions by Transformer architecture in image classification.
Compression Coding in Vision Consider the advantage of low-dimension representation and less computation cost, some recent works have attempted to introduce compression coding methods, like Sparse Coding [41], Principal Component Analysis (PCA) [42] and Discrete Cosine Transform (DCT) [43], into computer vision field. For image classification, [44] takes the DCT coefficients obtained from RGB images as the inputs of convolutional neural networks (CNNs) to reduce the communication bandwidth between CPU and GPU. DCT-Mask [45], based on Mask R-CNN, employs DCT supervision to produce high-quality mask representation. Analogously, [46] performed semantic segmentation on the DCT representation and fed the rearranged DCT coefficients to CNNs. MEInst [21] and ISTR [30] encode binary masks into fixed-dimensional mask vectors produced by PCA.

heading: 3 Method

heading: 3.1 Reviewing DETR
text: Recently, DETR [1] succeed in object detection. It formulates object detection as a set prediction problem and introduces object queries, a set of learnable embeddings, to represent objects. In DETR, each object query predicts an object for a given input image and set prediction loss is adopted to achieve one-to-one matching between the predicted and ground-truth objects in training phase. Further, the transformer encoder-decoder architecture is employed to model the relation between query embeddings and instances for better one-to-one set prediction.
Set Prediction Loss Let y = (c, b) and ŷ = (ĉ, b̂) denote the ground truths y and the set of predictions ŷ, respectively. c, ĉ ∈ RJ×S are the corresponding class labels and predicted class scores, where J and S are the object number and class number. b, b̂ ∈ RJ×4 are the corresponding ground-truth and predicted box coordinates. ω ∈ ΩJ is the assignment between the ground truths and predictions. Then the optimal one-to-one assignment ω∗ can be calculated by bipartite matching [47] as:
ω∗ = argmin ω∈ΩJ Ldet(y, ω(ŷ)) (1)
where the bipartite matching loss for object detection Ldet can be summarised as:
Ldet(y, ŷ) = λcls · Lcls(c, ĉ) + λL1 · LL1(b, b̂) + λgiou · Lgiou(b, b̂) (2) Here Lcls denotes the focal loss [48] for classifications, LL1 and Lgiou are L1 loss and generalized IoU loss [49] for box coordinates, respectively. λcls, λL1 and λgiou are corresponding coefficients.
Extension on Segmentation As shown in Fig. 1(a), DETR is further generalized to panoptic segmentation task by adding a multi-head attention (MHA) and FPN-style CNN after the Transformer decoder. Features from the encoder and learned query embeddings from the decoder are reshaped to spatial domain and then interact in MHA. The produced features are then gradually upsampled to the image size by the FPN-Style CNN to obtain spatial masks. It’s easy to adapt this framework to perform instance segmentation by cropping instance masks within detected bounding boxes.
Object Representation In DETR, objects are represented as a set of object queries. Object queries are initialized by the learnable query embeddings and then interact with the image features in the transformer decoder to update their representation. Finally, object query, as an unified representation, is directly used to classify and localize objects. However, the representation of spatial mask is built by
compulsorily reshaping the query embeddings to spatial domain in instance segmentation task. Such design leads to different representation forms compared to detection branch. Besides, the two-stage training process makes DETR fail to enjoy the benefit from multi-task learning.

heading: 3.2 Network Architecture
text: To encode the spatial mask into the end-to-end object detector in an unified form, we present a simple but efficient framework for instance segmentation based on DETR. The unified query representation (UQR) is proposed to perform instance-level localization and pixel-level segmentation simultaneously. The compression coding is further introduced to project the spatial mask into embedding domain for high-quality and efficient mask representation. The overall architecture of SOLQ is showed in Fig. 1(b). SOLQ can be divided into three parts: feature extraction network, Transformer decoder and unified query representation. We will describe our method in detail as follows.
Feature Extraction Network The feature extraction network consists of the backbone and Transformer encoder. Given an image I ∈ RH×W×3, ResNet [50] is used as the backbone to extract basic feature map x0 ∈ RC×H32×W32 , where H , W , C are the height, width and channels of feature map, respectively. Then the basis feature x0 is fed into K Transformer encoder layers to get the refined feature map xK ∈ RC×
HW 322 via {xk = Fk(xk−1)}Kk=1, iteratively. Each Transformer encoder layer
Fk(·) is composed of a multi-head self-attention (MHSA) and a feed-forward network (FFN). Transformer Decoder Given the learnable object queries, we generate the instance-aware query embeddings for the unified query representation by the Transformer decoder. In details, a set of learnable object queries q0 ∈ RJ×C are firstly randomly initialized. Then the initial object queries q0 interact with the refined feature map xK in K Transformer decoder layers to obtain instance-aware query embeddings qK ∈ RJ×C by {qk = Hk(qk−1, xK)}Kk=1. Each Transformer decoder layer Hk(·) has an extra multi-head cross-attention layer compared to the Transformer encoder layer. The instance-aware query embeddings qK are then fed into unified query representation part to generate predictions for three sub-tasks, including classification, localization and segmentation.
Unified Query Representation After Transformer decoder, each instance-aware query embedding in qK represents the features of corresponding instance. The supervision of three sub-tasks (classification, localization and segmentation) in an unified form (e.g. vector) is the last piece of the puzzle to achieve parallel predictions. Fig. 2 shows the learning of UQR. We mainly describes the joint supervision of multi-task learning as well as the training and inference processes of mask branch.
In details, UQR is learned under the supervision of classification, localization and mask branches. Both classification and localization branches are the same as in DETR [1]. The classification branch is a fully-connected (FC) layer that predicts the class confidences ĉ. The localization branch is a multi-layer perception (MLP) with hidden size 256 and predicts 4 box coordinates b̂. Similar to localization branch, the mask branch is also a multi-layer perception with hidden size 1024 and predicts mask vectors v̂ ∈ RJ×nk , nk is the dimension of each mask vector. During training, the mask
MLP
MLP
FC ℒ"#$
ℒ%&& ℒ()*+
𝑞-
Class vector
Box vector
Mask vector
Mask vector label Decoding
Encoding
ℒ./"
（Training）
（Inference）
𝑣
UQR: Unified Query Representation
Compression Coding
𝑚
2𝑚
?̂?
5𝑏
7𝑣
⋯
⋯
⋯MLP
MLP
FC ℒ"#$
ℒ%&& ℒ()*+
𝒒:
Class vector
Box vector
Mask vector
Mask vector label Decoding
Encoding
ℒ./"
（Training）
（Inference）
𝒗
UQR: Unified Query Representation
Compression Coding
𝒎
2𝒎
7𝒄
>𝒃
2𝒗
⋯
Figure 2: The learning of the proposed unified query representation. qK refers to the learned instance-aware query embeddings. ĉ, b̂, v̂ are predicted class, box and mask vectors, respectively. m̂ is the binary masks reconstructed. m and v denote ground-truth binary masks and mask vectors, correspondingly. Lcls, LL1&Lgiou and Lvec are the losses for classification, box regression and mask segmentation, respectively.
vectors predicted are supervised by the ground-truth mask vectors v ∈ RJ×nk generated from spatial mask m ∈ RJ×N×N by the mask compression coding described below, N is the spatial dimension of binary mask. While for the inference, the predicted mask vectors v̂ can be used to reconstruct the spatial masks m̂ by the inverse process of compression coding. Note that each Transformer decoder layer learns such an UQR and auxiliary supervision is adopted for better performance.
Mask Compression Coding As mentioned above, predicted mask vectors v̂ generated by the mask branch are supervised by the ground-truth mask vectors v. Here, we explore three compression coding methods to transform 2D spatial binary masks into 1D mask vectors, including Sparse Coding [41], Principal Component Analysis (PCA) [42] and Discrete Cosine Transform (DCT) [43].
Sparse Coding compresses the binary mask as a sparse combination of nk atoms from an overcomplete dictionary D ∈ Rnk×N2 . Ground-truth mask vectors v can be obtained from m through solving the minimum of Lasso [52] problem:
(v∗,D∗) = argmin (v,D)
( 1
2 ||m− vD||22 + β||v||1), s.t. ||De||2 = 1, ∀e ∈ [1, nk] (3)
where β is the regular coefficient. The binary masks m̂ can be reconstructed via m̂ = v̂D.
PCA transforms binary masks to low-dimensional mask vectors via matrix factorization. The process can be summarized as the following optimization problem:
P∗ = argmin P ||m−mPPT ||2, s.t. PPT = Unk (4)
where P ∈ RN2×nk and Unk ∈ Rnk×nk are the projection matrix and unit matrix, respectively. Ground-truth mask vectors v can be represented as v = mP meanwhile binary masks can be reconstructed by m̂ = v̂PT .
DCT first transforms ground-truth binary masks m into frequency domain according to f = AmAT , where A ∈ RN×N is the transform matrix and the (h, l) element of A can be calculated by Ah,l =√
1+sign(h) N cos[ (l+0.5)π N h]|. The ground-truth mask vectors v can be encoded by sampling the lowfrequency components from f . The binary masks m̂ can be recovered through the inverse sampling and transformation m̂ = A−1f̂(AT )−1, where f̂ is the inverse sampling result from v̂.
Loss Function We extend object ground-truths with mask vectors as y = (c, b, v) and corresponding predictions are ŷ = (ĉ, b̂, v̂). The overall loss function for supervision can be expressed as:
Linst = Ldet + λvec · Lvec(v, v̂) (5)
where Lvec is the mask vector loss and we use L1 loss in practice. λvec is the corresponding weight and Ldet is same as Eq. 2. Note that mask loss is not included in bipartite matching. The participation of mask loss may affect the global matching between the object queries and ground-truths.

heading: 3.3 Discussion
text: Whereas our SOLQ shares similarities with MEInst [21], DCT-Mask [45] and ISTR [30] in the compression coding of spatial mask, the main differences are described as follows. SOLQ produces the instance masks together with the instance class and the location in a parallel way. It aims to learn an unified query representation for better multi-task learning. The predictions of three sub-tasks are all obtained in a regression manner. Also, SOLQ is an end-to-end instance segmentation framework without any post-processes, like NMS. In contrast, DCT-Mask encodes the instance masks based on the RoI features cropped by the bounding boxes from detection branch. The mask encoding in MEInst and ISTR requires extra optimization process to obtain the optimal project matrix for reconstructing spatial masks. So they are not learned in an end-to-end manner. Also, our concurrent work, QueryInst [24] performs instance segmentation in an end-to-end fashion based on Sparse RCNN [31]. The overall architecture mainly follows ‘detect-then-segment’ paradigm. The features cropped by bounding boxes interact with the updated queries to generate the segmentation masks.

heading: 4 Experiments

heading: 4.1 Dataset and Metrics
text: We validate our method on COCO benchmark [19]. COCO contains 115k images for training, 5k for validation and 20k for testing, involving 80 object categories with instance-level segmentation annotations. We report results on COCO 2017 test-dev set for state-of-the-art comparison and the results on COCO 2017 val set for ablation studies. Consistent with Mask R-CNN [2], the standard COCO metrics including APbox, APboxS , AP box M , AP box L , and AP seg , APsegS , AP seg M , AP seg L are used to evaluate the performance of object detection and segmentation.

heading: 4.2 Implementation Details
text: For fast convergence, we build the SOLQ on top of Deformable DETR [35] in practice. ResNet [53], pretrained on the ImageNet [54] is employed as the backbone and multi-scale feature maps from C3 to C6 stages are used. For the deformable attention, the number of heads is set as 8 and the number of sampling points is set as 4. For the mask branch, nk is set to 256, hidden dim of MLP is 1024 and λvec is 3.0. Following DETR, λcls = 2, λL1 = 5, λgiou = 2. We train our model with Adam optimizer with weight decay of 1.0× 10−4. Models are trained for 50 epochs with the initial learning rate 2.0× 10−4 and decayed at 40th epoch by a factor 0.1. Multi-scale training is adopted, where the shorter side is randomly chosen within [408, 800] and the longer side is less or equal to 1333. All experiments are conducted over 8 Tesla V100 GPUs with batch size 32 except the comparison in Sec. 4.4. Since the D-DETR with SQR can only be trained with batch size 16, we also perform the D-DETR with UQR under the same setting for fair comparison.

heading: 4.3 Comparison with State-of-the-arts
text: As shown in Tab. 1, we compare SOLQ with state-of-the-art methods on COCO test-dev set. Our method achieves best performance on both APseg and APbox metrics. Compared to the typical twostage methods Mask R-CNN [2] and Cascade Mask R-CNN [7], SOLQ with ResNet-101 surpasses in APseg by 2.1% and 0.9%, respectively. Besides, we also compare SOLQ with state-of-the-art one-stage methods CondInst [9] and SOLOv2 [16], which are built based on dynamic convolution. Our method outperforms them 1.8% and 1.2% in APseg , respectively. Further, based on the recently introduced Swin Transformer [40], our SOLQ will serve as a fully-Transformer framework for instance segmentation and it can achieve 46.7% APseg and 56.5% APbox. To further validate the quality of boundary prediction, we also evaluate SOLQ using the Boundary AP [55] in Appendix A.1.
It is worthy noting that SOLQ performs well on objects of different scales, especially on small and middle scale objects. For example, SOLQ with ResNet-101 surpasses SOLOv2 by 5.2% in APsegS and 0.9% in APsegM , respectively. It should be owing to the mask compression encoding of spatial binary mask. In SOLOv2, the mask predictions are supervised by the ground-truth mask of 1/4 instance size so the performance of small objects are not optimized well. For our SOLQ, the mask
compression encoding encodes the high-resolution binary mask (e.g. 128× 128) into low-dimension mask vectors using the sparsity characteristic of the binary mask and keeps the principle information.
4.4 UQR vs. SQR
In this section, we show the comparison between the separate query representation (SQR) (see Fig. 1(a)) in DETR and the UQR used in SOLQ (see Fig. 1(b)). As shown in Tab. 2, we surprisingly find that UQR in SOLQ can boost the detection performance of DETR with a great margin, improving APbox by 2.3% and 2.0% with ResNet50 and ResNet101. While in comparison, separate query representation (SQR) only improve the APbox by 0.1% and the segmentation performance with 33.4 APseg is much lower than that of UQR. The large improvement in APbox shows the effectiveness of our proposed UQR, owing to the unified learning of query representation. For efficiency comparison between them, please refer to the Appendix A.2.
We also report the detection performance of both Faster R-CNN and Mask R-CNN since Mask R-CNN is built on top of Faster R-CNN. Compared to Faster R-CNN, Mask R-CNN improves APbox by 0.6% and 1.1% with ResNet50-FPN and ResNet101-FPN, respectively. As we see, multi-task learning tends to improve the performance with each other and the performance can be further improved if these sub-tasks are learned using an unified representation. SOLQ learns the UQR to perform classification, localization and segmentation simultaneously in a regression manner. For both SQR and Mask R-CNN, full-connected layers are employed to classify the objects and regress box coordinates while the mask generated by full convolution network is supervised by 2D spatial mask.
Further, Fig. 3 shows the visualization comparison between the DETR with SQR and our SOLQ. Overall, SOLQ generates much more fine-grained masks and provides better object detection performance. We also show some failure cases in occluded environments (see Appendix A.5).

heading: 4.5 Ablation Studies
text: In this section, we also ablate several critical factors in SOLQ by progressively adjusting each factor in the system. It can be seen that each factor contributes to the final success of the SOLQ. Note that
Fa ile d C
as
e
our ablation experiments are conducted using the single feature level with C5 and validated on the COCO 2017 val set.
Mask Compression Coding Methods We compare the impact of different compression coding methods to encode binary masks. As shown in Tab. 3a, flatting the spatial masks into 1D mask vectors directly for supervision can obtain decent segmentation result while improving APbox by 1%. DCT produce the best performance in both APseg and APbox metrics. The main reason is that the loss of mask compression caused by DCT is relatively small, compared to the Sparse Coding and PCA methods. Also, DCT can be employed for each image in an online manner while Sparse Coding and PCA are performed on the whole training set to get "principal components" or "dictionary" in an offline way. Therefore, we choose DCT as our default method for mask compression coding.
Mask Vector Loss Weight Tab. 3b shows the effect of adjusting the weight of mask vector loss. When the weight of mask vector loss λvec is set as 3.0, SOLQ gets the best performance in both APseg and APbox. The regression dimension of DCT vector (e.g. 256) is usually larger than that of the bounding box (e.g. 4), so the magnitude of mask vector loss is larger than that of the regression loss. Therefore, an appropriate λvec can keep the balance between the mask branch and localization branch such that these two branches can be jointly optimized better.
Mask Vector Loss Stage We also ablate the impact of the number of decoder stages enabling mask vector loss in Tab. 3c. It can be seen that adding auxiliary mask vector loss on all decoders can improve by 2.9% and 1.1% on APseg and APbox, respectively. The experimental results show that auxiliary loss employed in multiple decoders is helpful to both mask and detection branches. Note
that auxiliary loss of detection branch is enabled in all decoder stages in this ablation, so the gain of detection branch is not as large as the gain of the mask branch.
Spatial Resolution of Binary Mask As mentioned above, mask compression coding projects the 2D N ×N ground-truth binary masks into 1D mask vectors for supervision. In Tab. 3d, we explore the impact of the spatial resolution of ground truth binary mask. The segmentation performance APseg is improved from 30.8 to 32.6 as N increases from 64 to 128 and APseg no longer improves when N is greater than 128. Ground-truth binary mask with high resolution can keep more instance details but needs high-dimension mask vectors to reconstruct. Therefore, once the number of coefficients nk is chosen, N has the most suitable value corresponding to nk.
Dimension of Mask Vector Similar to the spatial resolution of ground truth binary mask, there is also a suitable value nk for given N . Tab. 3e shows that nk = 256 achieves the best performance in both APseg and APbox when the spatial resolution N is set as 128. Theoretically, mask vector with higher dimension should reconstruct a better binary mask. However, high-dimension mask vector will enlarge the regression dimension, making it hard to optimize the mask branch. As a result, the quality of the binary mask reconstructed reduces a lot.

heading: 5 Conclusion
text: In this paper, we present SOLQ, a new instance segmentation framework. Based on DETR, SOLQ learns an unified query representation, which is used to predict the instance masks parallel with object detection in an end-to-end manner. To make the mask representation consistent with the query embedding, SOLQ projects the high-resolution spatial masks into low-dimensional mask vectors and regards mask prediction as the regression of mask vectors. SOLQ is a truly one shot framework without any two-stage operations, like ROIAlign. On the challenging COCO dataset, SOLQ achieves state-of-the-art performance on instance segmentation task and greatly improves the detection performance of DETR thanks to the multi-task learning. We believe that SOLQ will serve as a strong baseline for instance segmentation for its excellent performance and simplicity.

Review summaries:
The paper proposes a different approach to instance segmentation using DETR-like architecture. Instead of the FPN-based approach adopted in the original DETR, they opt for regressing directly an encoded version of the mask.  This approach allows to perform segmentation in a cheaper manner, and also allows to use the segmentation objective in the auxiliary loss (not possible in original DETR due to prohibitive cost), which gives some substantial improvement.  The results shown on Coco are competitive with current state of the art. 

The paper proposes and end-to-end instance segmentation approach based on recently proposed Transformer-based object detector DETR. In DETR instance segmentation is done by upsampling of ResNet features from multiple stages, which is computationally expensive. Instead, the authors propose to predict instance masks together with bounding box and class labels, by compressing the masks. This results in a simple and efficient detection and instance segmentation model. Moreover, AP on both tasks improves significantly over DETR and Deformable DETR, achieving very impressive results on par with state-of-the-art, on the challenging COCO dataset. The paper also provides a detailed ablation study of the proposed model.

This paper tries to combine DETR and Mask Encoding techniques for end-to-end instance segmentation: DETR could directly separate different objects and Mask Encoding could encode the arbitrary instance mask into a low-dimensional and fix representation. To verify the effectiveness of each component, the authors conducted extensive ablations for each component. The final performance is inspiring, espically using Trasnformer backbone.