SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Training. We use mmdetection<sup>3</sup> as codebase and follow the default training settings. For a fair comparison, we follow the training protocol in [17]: 1) the number of instances centers is set to 100; 2) a linear combination of the  $\mathcal{L}_1$  loss and the GIoU Loss is used as the optimization objective for bounding box regression. Their coefficients are set to 5 and 2, respectively. In addition, the final object centers are fed into a small FFN for object classification, trained with a binary cross-entropy loss. Moreover, we set the initial learning rate to  $1 \times 10^{-5}$ , the training epoch to 50, and the batch size to 16. We use random scale jittering with a factor in [0.1, 2.0] and a crop size of  $1024 \times 1024$ .
Test. We use one input image scale with shorter side as 800.
Metric. We adopt AP,  $\mathrm{AP}_{50}$ ,  $\mathrm{AP}_{75}$ ,  $\mathrm{AP}_S$ ,  $\mathrm{AP}_M$ , and  $\mathrm{AP}_L$ .
Performance Comparison. In Table 2, we present the numerical results for CLUSTERFORMER for object detection. We observe that it surpasses all counterparts [70, 7, 56, 77, 9, 75, 60, 102, 71, 96] with remarkable gains with respect to mAP. In particular, CLUSTERFORMER-Tiny exceeds the vanilla Deformable DETR [102], Sparse-DETR [71], and DINO [96] over Swin-T [53] by  $6.5\%$ ,  $3.4\%$ , and  $0.8\%$  in terms of mAP, respectively. In addition, our approach also outperforms these methods over Swin-S [53], i.e.,  $54.2\%$  vs  $48.3\%$  vs  $49.9\%$  vs  $53.3\%$  in terms of mAP, respectively. Notably, CLUSTERFORMER achieves impressive performance without relying on additional augmentation.
4.1.3 Experiments on Semantic Segmentation
Training. We use mmsegmentation<sup>4</sup> as codebase and follow the default training settings. The training process for semantic segmentation involves setting the number of cluster centers to match the number of semantic categories, which is 150 for ADE20K [101]. Following the approach employed in recent works [97, 17, 74], we adopt a combination of the standard cross-entropy loss and an auxiliary dice loss for the loss function. By default, the coefficients for the cross-entropy and dice losses are set to 5 and 1, respectively. In addition, we configure the initial learning rate to  $1 \times 10^{-5}$ , the number of training epochs to 50, and the batch size to 16.
$^{2}$ https://github.com/open-mmlab/mmclassification
<sup>3</sup>https://github.com/open-mmlab/mmdetection
$^{4}$ https://github.com/open-mmlab/mmsegmentation
Table 2: Quantitative results on COCO [49] test-dev for object detection (see §4.1.2 for details).
Algorithm	Backbone	Epoch	mAP↑	AP50↑	AP75↑	APS	APM↑	APL↑
Faster R-CNN[NeuIPS15] [70]	ResNet-101	36	41.7	62.3	45.7	24.7	46.0	53.2
Cascade R-CNN[CVPR18] [7]	ResNet-101	36	42.8	61.1	46.7	24.9	46.5	56.4
Grid R-CNN[CVPR19] [56]	ResNet-50	24	40.4	58.5	43.6	22.7	43.9	53.0
EfficientDet[CVPR20] [77]	Efficient-B3	300	45.4	63.9	49.3	27.1	49.5	61.3
DETR[ECCV20] [9]	ResNet-50	150	39.9	60.4	41.7	17.6	43.4	59.4
Sparse R-CNN[CVPR21] [75]	ResNet-101	36	46.2	65.1	50.4	29.5	49.2	61.7
Conditional DETR[ICCV21] [60]	ResNet-50	50	41.1	61.9	43.5	20.4	44.5	59.9
Deformable DETR[ICLR21] [102]	Swin-T	50	45.5±0.26	65.2±0.20	49.8±0.21	27.0±0.26	49.1±0.24	60.7±0.29
Swin-S	48.3±0.21	68.7±0.27	52.1±0.27	30.5±0.28	51.6±0.22	64.4±0.19
Sparse-DETR[ICLR22] [71]	Swin-T	50	48.6±0.24	69.6±0.20	53.5±0.23	30.1±0.27	51.8±0.21	64.9±0.29
Swin-S	49.9±0.21	70.3±0.27	54.0±0.26	32.5±0.22	53.6±0.28	66.2±0.25
DINO[ICLR23] [96]	Swin-T	50	51.2±0.26	68.4±0.25	55.3±0.26	31.3±0.24	55.1±0.38	65.8±0.26
Swin-S	53.3±0.27	70.9±0.38	57.6±0.23	33.8±0.23	56.4±0.32	66.9±0.26
CLUSTERFORMER	Ours-Tiny	50	52.0±0.32	70.4±0.25	57.5±0.32	34.2±0.28	54.8±0.29	64.8±0.22
Ours-Small	54.2±0.33	71.8±0.16	59.1±0.17	35.6±0.28	57.2±0.20	67.4±0.18
Furthermore, we employ random scale jittering, applying a factor within the range of [0.5, 2.0], and utilize a crop size with a fixed resolution of  $640 \times 640$  pixels.
Test. During the testing phase, we re-scale the input image with a shorter side to 640 pixels without applying any additional data augmentation at test time.
Metric. Mean intersection-over-union (mIoU) is used for assessing image semantic segmentation performance.
Performance Comparison. Table 3 shows the results on semantic segmentation. Empriailly, our method compares favorably to
recent transformer-based approaches [54, 15, 32, 100, 74, 90, 95, 17]. For instance, CLUSTER-FORMER-Tiny surpasses both recent advancements, i.e., kMaX-Deeplab [95] and Mask2Former [17] with Swin-T [53] (i.e.,  $49.1\%$  vs.  $48.3\%$  vs.  $48.5\%$ ), respectively. Moreover, CLUSTERFORMER-Small achieves  $52.4\%$  mIoU and outperforms all other methods in terms of mIoU, making it competitive with state-of-the-art methods as well.
Table 3: Quantitative results on ADE20K [101] val for semantic segmentation (see §4.1.3 for details).
Algorithm	Backbone	Epoch	mIoU↑
FCN[CVPR2015][54]	ResNet-50	50	36.0
DeeplabV3+[ECCV2018][15]	ResNet-50	50	42.7
APCNet[CVPR2019][32]	ResNet-50	100	43.4
SETR[CVPR2021][100]	ViT-L	100	49.3
Segmenter[ICCV2021][74]	ViT-B	100	52.1
Segformer[NeurIPS2021][90]	MIT-B5	100	51.4
kMaX-Deeplab[ECCV2022][95]	ConvNeXt-T	100	48.3±0.15
ConvNeXt-S	51.6±0.23
Mask2Former[CVPR2022][17]	Swin-T	100	48.5±0.24
Swin-S	51.1±0.21
CLUSTERFORMER	Ours-Tiny	100	49.1±0.19
Ours-Small	52.4±0.23
4.1.4 Experiments on Instance Segmentation
Training. We adopt the same training strategy for instance segmentation by following §4.1.2. For instance segmentation, we change the training objective by utilizing a combination of the binary cross-entropy loss and the dice Loss for instance mask optimization.
Test. We use one input image scale with a shorter side of 800.
Performance Comparison. Table 4 presents the results of CLUSTERFORMER against famous instance segmentation methods [33, 8, 14, 43, 13, 26, 23, 18, 17, 44] on COCO test-dev. CLUSTERFORMER shows clear performance advantages over prior arts. For example, CLUSTERFORMER-Tiny outperforms the universal counterparts Mask2Former [17] by  $1.4\%$  over Swin-T [53] in terms of mAP and on par with the state-of-the-art method, Mask-Dino [44] with Swin-T backbone. Moreover, CLUSTERFORMER-Small surpasses all the competitors, e.g., yielding significant gains of  $1.0\%$  and  $0.5\%$  mAP compared to Mask2Former and Mask-Dino over Swin-S, respectively. Without bells and whistles, our method establishes a new state-of-the-art on COCO instance segmentation.
4.1.5 Experiments on Panoptic Segmentation
Training. Following the convention [84, 17], we use the following objective for network learning:
$$
\mathcal {L} ^ {\text {P a n o p t i c}} = \lambda^ {\text {t h}} \mathcal {L} ^ {\text {t h}} + \lambda^ {\text {s t}} \mathcal {L} ^ {\text {s t}} + \lambda^ {\text {a u x}} \mathcal {L} ^ {\text {a u x}}, \tag {7}
$$
$\mathcal{L}^{\mathrm{th}}$  and  $\mathcal{L}^{\mathrm{st}}$  represent the loss functions for things and stuff, respectively. To ensure a fair comparison, we follow [95, 85] and incorporate an auxiliary loss calculated as a weighted sum of four different loss terms, specifically, a PQ-style loss, a mask-ID cross-entropy loss, an instance discrimination
Table 4: Quantitative results on COCO [49] test-dev for instance segmentation (see §4.1.4 for details).
Algorithm	Backbone	Epoch	mAP↑	AP50↑	AP75↑	APS	APM↑	APL↑
Mask R-CNN[1CCV2017][33]	ResNet-101	12	36.1	57.5	38.6	18.8	39.7	49.5
Cascade MR-CNN[PAM2019][8]	ResNet-101	12	37.3	58.2	40.1	19.7	40.6	51.5
HTC[CVPR2019][14]	ResNet-101	20	39.6	61.0	42.8	21.3	42.9	55.0
PointRend[CVPR2020][43]	ResNet-50	12	36.3	56.9	38.7	19.8	39.4	48.5
BlendMask[CVPR2020][13]	ResNet-101	36	38.4	60.7	41.3	18.2	41.5	53.3
QueryInst[CVCCV2021][26]	ResNet-101	36	41.0	63.3	44.5	21.7	44.4	60.7
SOLQ[NeuIPS2021][23]	Swin-L†	50	46.7	72.7	50.6	29.2	50.1	60.9
SparseInst[CVPR2022][18]	ResNet-50	36	37.9	59.2	40.2	15.7	39.4	56.9
Mask2Former[CVPR2022][17]	Swin-T	50	44.5±0.16	67.3±0.15	47.7±0.24	23.9±0.20	48.1±0.16	66.4±0.15
Swin-S	46.0±0.21	68.4±0.22	49.8±0.24	25.4±0.19	49.7±0.22	67.4±0.24
Mask-Dino[CVPR2023][44]	Swin-T	50	45.8±0.28	69.6±0.29	50.2±0.26	26.0±0.28	48.7±0.37	66.4±0.29
Swin-S	46.5±0.39	70.1±0.34	52.2±0.28	27.6±0.34	49.9±0.25	69.5±0.29
CLUSTERFORMER	Ours-Tiny	50	45.9±0.26	69.1±0.21	49.5±0.18	25.2±0.22	50.1±0.24	68.8±0.24
Ours-Small	47.0±0.19	71.5±0.26	51.8±0.24	27.3±0.16	50.5±0.20	72.6±0.22
Table 5: Quantitative results on COCO Panoptic [42] val for panoptic segmentation (see §4.1.5 for details).
Algorithm	Backbone	Epoch	\( PQ^{\dagger} \)	\( PQ^{th}\uparrow \)	\( PQ^{st}\uparrow \)	\( mAP_{pan}^{th}\uparrow \)	mIoUpan↑
Panoptic-FPN[CVPR2019][41]	ResNet-101	20	44.0	52.0	31.9	34.0	51.5
UPSNet[CVPR2019][92]	ResNet-101	12	46.2	52.8	36.5	36.3	56.9
Panoptic-Deeplab[CVPR2020][16]	Xception-71	12	41.2	44.9	35.7	31.5	55.4
Panoptic-FCN[CVPR2021][45]	ResNet-50	12	44.3	50.0	35.6	35.5	55.0
Max-Deeplab[CVPR2021][85]	Max-L	55	51.1	57.0	42.2	-	-
CMT-Deeplab[CVPR2022][94]	Axial-R104†	55	54.1	58.8	47.1	-	-
Panoptic Segformer[CVPR2022][46]	ResNet-50	24	49.6±0.25	54.4±0.26	42.4±0.25	39.5±0.20	60.8±0.21
ResNet-101	50.6±0.21	55.5±0.24	43.2±0.20	40.4±0.21	62.0±0.22
Mask2Former[CVPR2022][17]	Swin-T	50	53.2±0.25	59.1±0.22	43.3±0.23	42.3±0.27	62.9±0.19
Swin-S	54.1±0.29	60.2±0.28	45.6±0.18	43.1±0.23	63.6±0.31
Mask Dino[CVPR2023][44]	Swin-T	50	53.6±0.29	59.5±0.26	44.0±0.24	44.3±0.29	63.2±0.27
Swin-S	54.9±0.33	61.1±0.23	46.2±0.26	45.0±0.22	64.3±0.30
CLUSTERFORMER	Ours-Tiny	50	54.7±0.22	60.8±0.31	46.1±0.20	43.4±0.25	64.0±0.20
Ours-Small	55.8±0.38	61.9±0.39	47.2±0.23	44.2±0.22	65.5±0.21
loss, and a semantic segmentation loss. More information about  $\mathcal{L}^{\mathrm{aux}}$  can be found in [85, 95]. The coefficients  $\lambda^{\mathrm{th}},\lambda^{\mathrm{st}}$  , and  $\lambda^{\mathrm{aux}}$  are assigned the values of 5, 3, and 1, respectively. Furthermore, the final centers are input into a small feed-forward neural network (FFN) for semantic classification, which is trained using a binary cross-entropy loss. Moreover, we set the initial learning rate to  $1\times 10^{-5}$  , the number of training epochs to 50, and the batch size to 16. We also employ random scale jittering with a factor range of [0.1, 2.0] and a crop size of  $1024\times 1024$
Metric. We employ the PQ metric [42] and report  $\mathbf{PQ}^{\mathrm{Th}}$  and  $\mathbf{PQ}^{\mathrm{St}}$  for the "thing" and "stuff" classes, respectively. To ensure comprehensiveness, we also include  $\mathbf{mAP}_{\mathrm{pan}}^{\mathrm{Th}}$ , which evaluates mean average precision on "thing" classes using instance segmentation annotations, and  $\mathbf{mIoU}_{\mathrm{pan}}$ , which calculates mIoU for semantic segmentation by merging instance masks belonging to the same category, using the same model trained for the panoptic segmentation task.
Performance Comparison. We perform a comprehensive comparison against two divergent groups of state-of-the-art methods: universal approaches [46, 17, 44] and specialized panoptic methods [41, 92, 16, 45, 85, 97, 94]. As shown in Table 5, CLUSTERFORMER outperforms both types of rivals. For instance, the performance of CLUSTERFORMER-Tiny clear ahead compared to Mask2Former [17] (i.e.,  $54.7\%$  PQ vs.  $53.2\%$  PQ) and Mask-Dino [44] (i.e.,  $54.7\%$  PQ vs.  $53.6\%$  PQ) on the top of Swin-T [53], and CLUSTERFORMER-Small achieves promising gains of  $1.7\%$  and  $0.9\%$  PQ against Mask2Former and Mask-Dino over Swin-S, respectively. Moreover, in terms of  $\mathrm{mAP}_{\mathrm{pan}}^{\mathrm{Th}}$  and  $\mathrm{mIoU}_{\mathrm{pan}}$ , the CLUSTERFORMER also achieves outstanding performance beyond counterpart approaches.
4.2 Ablative Study
This section ablates CLUSTERFORMER's key components on ImageNet [72] and MS COCO [49] validation split. All experiments use the tiny model.
Key Component Analysis. We first investigate the two major elements of CLUSTERFORMER, specifically, Recurrent Cross-Attention Clustering for center updating and Feature Dispatching for feature updating. We construct a BASELINE model without any center updating and feature dispatching technique. As shown in Table 6a, BASELINE achieves  $74.59\%$  top-1 and  $91.73\%$  top-5 accuracy. Upon applying Recurrent Cross-Attention Clustering to the BASELINE, we observe consistent and substantial improvements for both top-1 accuracy  $(74.59\% \rightarrow 80.57\%)$  and top-5 accuracy  $(91.73\% \rightarrow 95.22\%)$ . This highlights the importance of the center updating strategy
Table 6: A set of ablative studies on ImageNet [72] validation and MS COCO [49] test-dev split (see §4.2). The adopted designs are marked in red.
Algorithm Component	#Params	top-1	top-5
BASELINE	21.73M	74.59	91.73
+ Recurrent Cross-Attention Clustering	26.27M	80.57	95.22
+ Feature Dispatching	23.46M	78.58	94.68
CLUSTERFORMER (both)	27.85M	81.31	96.32
(a) Key Component Analysiss
Numbers (T)	#Params	top-1	top-5
1	27.85M	81.06	96.23
2	81.22	96.29
3	81.31	96.32
4	81.33	96.33
(b) Number of Recursion
Variant Cluster Center Updating Strategy	#Params	top-1	top-5
Cosine Similarity	23.88M	78.79	94.36
Vanilla Cross-Attention [81]	35.48M	79.67	94.95
Criss Cross-Attention [35]	34.16M	79.91	95.24
K-Means [95]	27.71M	80.96	95.57
Recurrent Cross-Attention	27.85M	81.31	96.32
(c) Recurrent Cross-Attention Clustering
Head Dimension	#Params	top-1	top-5
16	17.25M	71.69	90.16
24	22.88M	75.37	92.45
32	27.85M	81.31	96.32
40	32.81M	82.21	97.09
48	38.14M	82.40	97.22
(d) Head Dimension
Feature Dispatching	#Params	top-1	top-5
None	26.27M	80.57	95.22
Vanilla FC Layer	27.14M	80.83	95.47
Confidence-Based [68]	26.81M	80.69	95.30
FC w/ Similarity [58]	27.46M	80.96	95.84
Ours (Eq. 6)	27.85M	81.31	96.32
(e) Feature Dispatching
Decoder Query Initialization	mAP↑	AP50↑	AP75↑
Free Parameters	44.2	66.3	46.4
Direct Feature Embedding [17]	44.5	67.3	47.2
Mixed Query Selection [44]	44.9	67.9	47.8
Scene-Adoptive Embedding [47]	45.1	67.8	48.0
Centers from Encoder (Ours)	45.9	69.1	49.5
(f) Decoder Query Initialization for instance segmentation
and validates the effectiveness of our approach, even without explicitly performing clustering. Furthermore, after incorporating Feature Dispatching into the BASELINE, we achieve significant gains of  $3.99\%$  in top-1 accuracy and  $2.95\%$  in top-5 accuracy. Finally, by integrating both core techniques, CLUSTERFORMER delivers the best performance across both metrics. This indicates that the proposed Recurrent Cross-Attention Clustering and Feature Dispatching can work synergistically and validates the effectiveness of our comprehensive algorithmic design.
Recurrent Cross-attention Clustering. We next study the impact of our Recurrent Cross-attention Clustering (Eq.4) by contrasting it with the cosine similarity updating, basic cross-attention [81], Criss-attention [35] and  $K$ -Means cross-attention [95]. As illustrated in Table 6c, our Recurrent Cross-Attention proves to be effective – it outperforms the cosine similarity, vanilla, Criss and  $K$ -Means by  $2.52\%$ ,  $1.64\%$ ,  $1.40\%$  and  $0.15\%$  top-1 accuracy respectively, and efficient – its #Params are significantly less than the other vanilla and Criss-attention and on par with  $K$ -Means, in line with our analysis in §3.2. To gain further insights into recursive clustering, we examine the effect of the recursion number  $T$  in Table 6b. We discover that performance progressively improves from  $81.06\%$  to  $81.31\%$  in top-1 accuracy when increasing  $T$  from 1 to 3, but remains constant after running additional iterations. We also observe that #Params increase as  $T$  increases. Consequently, we set  $T = 3$  as the default to strike an optimal balance between accuracy and computation cost.
Multi-head Dimension. We then ablate the head embedding dimension for the attention head in Table 6d. We find that performance significantly improves from  $71.69\%$  to  $82.40\%$  in top-1 accuracy when increasing the dimension from 16 to 48, but #Params steadily increase as the dimension grows. For a fair comparison with Swin [53], we set the head dimension to 32 as our default.
Feature Dispatching. We further analyze the influence of our Feature Dispatching. As outlined in Table 6e, in a standard manner without any dispatching method, the model attains  $80.57\%$  top-1 accuracy and  $95.22\%$  top-5 accuracy. By applying a vanilla fully connected layer to update the feature, we witness a marginal increase of  $0.26\%$  in top-1 accuracy. Moreover, using the confidence-based updating method [68] and fully connected layer with similarity, the model demonstrates a noticeable enhancement in  $0.12\%$  and  $0.39\%$  top-1 accuracy, respectively. Last, our method yields significant performance advancements across both metrics, i.e.,  $81.31\%$  top-1 and  $96.32\%$  top-5 accuracy.
Decoder Query Initialization. Last, we examine the impact of query initialization in the decoder on a downstream task (i.e., instance segmentation) in Table 6f. For free parameter initialization, the base model can achieve  $44.2\%$  in terms of mAP. By applying direct feature embedding, the method has a slight improvement of  $0.3\%$  mAP. In addition, the model exhibits improvements in
Figure 3: Visualization of center-feature assignment at the last stage of recurrent cross-attention clustering with the resolution of 7 by 7. The map displays distinct clusters, each containing features with similar representations. mAP, achieving  $44.9\%$  and  $45.1\%$ , respectively, by employing the mixed query selection [44] and scene-adoptive embedding [47]. Outstandingly, CLUSTERFORMER achieves the highest performance in all three metrics, i.e.,  $45.9\%$  mAP,  $69.1\%$ $\mathrm{AP}_{50}$  and  $49.5\%$ $\mathrm{AP}_{75}$ , respectively. The empirical evidence proves our design — using the cluster centers from the encoder to derive the initial query for the decoder — that facilitates the transferability for representation learning.
Ad-hoc Explainability. We visualize the cluster assignment map for image classification in Fig. 3. This figure provides an insightful illustration of how CLUSTERFORMER groups similar features together. Each color represents a cluster of features that share common characteristics.
This study adopts an epistemological perspective centered on the clustering-based paradigm, which advocates a universal vision framework named CLUSTERFORMER. This framework aims to address diverse visual tasks with varying degrees of clustering granularity. By leveraging insights from clustering, we customize the cross-attention mechanism for recursive clustering and introduce a novel method for feature dispatching. Empirical findings provide substantial evidence to support the effectiveness of this systematic approach. Based on its efficacy, we argue deductively that the proposed universal solution will have a substantial impact on the wider range of visual tasks when viewed through the lens of clustering. This question remains open for our future endeavors.
Acknowledgement. This research was supported by the National Science Foundation under Grant No. 2242243.
[1] Sameer Agarwal, Jongwoo Lim, Lihi Zelnik-Manor, Pietro Perona, David Kriegman, and Serge Belongie. Beyond pairwise clustering. In CVPR, 2005.
[2] Merav Ahissar and Shaul Hochstein. The reverse hierarchy theory of visual perceptual learning. Trends in cognitive sciences, 8(10):457-464, 2004.
[3] Valerie Ahl and Timothy FH Allen. *Hierarchy theory: a vision, vocabulary, and epistemology*. Columbia University Press, 1996.
[4] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021.
[5] Gijs Joost Brouwer and David J Heeger. Categorical clustering of the neural representation of color. Journal of Neuroscience, 33(39):15454-15465, 2013.
[6] Xiao Cai, Feiping Nie, Heng Huang, and Farhad Kamangar. Heterogeneous image feature integration via multi-modal spectral clustering. In CVPR, 2011.
[7] Zhaowei Cai and Nuno Vasconcelos. Cascade r-cnn: Delving into high quality object detection. In CVPR, 2018.
[8] Zhaowei Cai and Nuno Vasconcelos. Cascade r-cnn: high quality object detection and instance segmentation. IEEE TPAMI, 43(5):1483-1498, 2019.
[9] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV, 2020.
[10] Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. Deep clustering for unsupervised learning of visual features. In ECCV, 2018.
[11] M Emre Celebi. *Partitional clustering algorithms*. Springer, 2014.
[12] Antoni B Chan and Nuno Vasconcelos. Modeling, clustering, and segmenting video with mixtures of dynamic textures. IEEE TPAMI, 30(5):909-926, 2008.
[13] Hao Chen, Kunyang Sun, Zhi Tian, Chunhua Shen, Yongming Huang, and Youliang Yan. Blendmask: Top-down meets bottom-up for instance segmentation. In CVPR, 2020.
panoptic segmentation. In CVPR, 2022.
[95] Qihang Yu, Huiyu Wang, Siyuan Qiao, Maxwell Collins, Yukun Zhu, Hatwig Adam, Alan Yuille, and Liang-Chieh Chen. k-means mask transformer. ECCV, 2022.
[96] Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605, 2022.
[97] Wenwei Zhang, Jiangmiao Pang, Kai Chen, and Chen Change Loy. K-net: Towards unified image segmentation. NeurIPS, 2021.
[98] Ying Zhao, George Karypis, and Usama Fayyad. Hierarchical clustering algorithms for document datasets. Data mining and knowledge discovery, 10:141-168, 2005.
[99] Minghang Zheng, Peng Gao, Renrui Zhang, Kunchang Li, Xiaogang Wang, Hongsheng Li, and Hao Dong. End-to-end object detection with adaptive clustering transformer. BMVC, 2021.
[100] Sixiao Zheng, Jiachen Lu, Hengshuang Zhao, Xiatian Zhu, Zekun Luo, Yabiao Wang, Yanwei Fu, Jianfeng Feng, Tao Xiang, Philip HS Torr, et al. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. In CVPR, 2021.
[101] Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. In CVPR, 2017.
[102] Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. In ICLR, 2021.