SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Training loss. Our architecture consists of a generative part, i.e., object mining transformer  $G$  and a discriminative part, i.e., detail mining transformer  $D$ . The  $D$  judges whether the mask is real (ground truth) or fake (generated by  $G$ ) by mining the detailed features of the object framed by the masks. To fool  $D$ ,  $G$  is supposed to predict a more accurate mask approaching the ground truth. We alternately train two parts to achieve mutual promotion. When training  $D$ , the parameters of  $G$  are frozen, and the loss function for  $D$  is formulated as:
$$
\mathcal {L} _ {d} = - \log \left(\mathbf {F C} \left(\boldsymbol {\omega} _ {r} ^ {k}\right)\right) - \log \left(1 - \mathbf {F C} \left(\boldsymbol {\omega} _ {f} ^ {k}\right)\right) + \lambda_ {d i v} \mathcal {L} _ {d i v}, \tag {13}
$$
where the  $\mathbf{FC}:\Omega \to [0,1]$  denotes the fully-connected layers that output the real/fake results, and the  $\omega_r^k,\omega_f^k\in \Omega$  denote the most different pair of local features from the object specified by ground truth  $\mathbf{M}_{gt}$  and the predicted mask  $\mathbf{M}_e$ , respectively.  $\lambda_{div}$  denotes the weight of diversity loss and we
Table 2: Comparison with other state-of-the-art methods for 1-shot and 5-shot segmentation on PASCAL-  ${5}^{i}$  using the mIoU (%) evaluation metric. Best results are shown in bold.
Method	Backbone	1-shot	5-shot
fold0	fold1	fold2	fold3	Mean	fold0	fold1	fold2	fold3	Mean
PFENet[TPAMI2020] [14]	61.7	69.5	55.4	56.3	60.8	63.1	70.7	55.8	57.9	61.9
RePRI[CVPR2021] [54]	59.8	68.3	62.1	48.5	59.7	64.6	71.4	71.1	59.3	66.6
HSNet[ICCV2021] [8]	64.3	70.7	60.3	60.5	64.0	70.3	73.2	67.4	67.1	69.5
CyCTR[NIPS2021] [7]	65.7	71.0	59.5	59.7	64.0	69.3	73.5	63.8	63.5	67.5
NERTNet[CVPR2022] [43]	65.4	72.3	59.4	59.8	64.2	66.2	72.8	61.7	62.2	65.7
DCAMA[ECCV2022][9]	67.5	72.3	59.6	59.0	64.6	70.5	73.9	63.7	65.8	68.5
SSP[ECCV2022] [11]	60.5	67.8	66.4	51.0	61.4	67.5	72.3	75.2	62.1	69.3
IPMT[NIPS2022] [6]	72.8	73.7	59.2	61.6	66.8	73.1	74.7	61.6	63.4	68.2
VAT[ECCV2022] [48]	67.6	72.0	62.3	60.1	65.5	72.4	73.6	68.6	65.7	70.1
BAM[CVPR2022] [55]	69.0	73.6	67.6	61.1	67.8	70.6	75.1	70.8	67.2	70.9
HDMNet[CVPR2023] [13]	71.0	75.4	68.9	62.1	69.4	71.3	76.2	71.3	68.5	71.8
AMFomer(ours)	71.1	75.9	69.7	63.7	70.1	73.2	77.8	73.2	68.7	73.2
PFENet[TPAMI2020] [14]	60.5	69.4	54.4	55.9	60.1	62.8	70.4	54.9	57.6	61.4
CyCTR[NIPS2021] [7]	67.2	71.1	57.6	59.0	63.7	71.0	75.0	58.5	65.0	67.4
NERTNet[CVPR2022] [43]	65.5	71.8	59.1	58.3	63.7	67.9	73.2	60.1	66.8	67.0
DCAMA[ECCV2022] [9]	65.4	71.4	63.2	58.3	64.6	70.7	73.7	66.8	61.9	68.3
SSP[ECCV2022] [11]	63.2	70.4	68.5	56.3	64.6	70.5	76.4	79.0	66.4	73.1
IPMT[NIPS2022] [6]	71.6	73.5	58.0	61.2	66.1	75.3	76.9	59.6	65.1	69.2
VAT[ECCV2022] [48]	70.0	72.5	64.8	64.2	67.9	75.0	75.2	68.4	69.5	72.0
AMFomer(ours)	71.3	76.7	70.7	63.9	70.7	74.4	78.5	74.3	67.2	73.6
Table 3: Comparison with other state-of-the-art methods for 1-shot and 5-shot segmentation on COCO-20 $^i$  using the mIoU (\%) evaluation metric. Best results are shown in bold.
Method	Backbone	1-shot	5-shot
fold0	fold1	fold2	fold3	Mean	fold0	fold1	fold2	fold3	Mean
NERTNet[CVPR2022] [43]	Res-101	38.3	40.4	39.5	38.1	39.1	42.3	44.4	44.2	41.7	43.2
SSP[ECCV2022] [11]	39.1	45.1	42.7	41.2	42.0	47.4	54.5	50.4	49.6	50.2
HSNet[ICCV2022] [8]	37.2	44.1	42.4	41.3	41.2	45.9	53.0	51.8	47.1	49.5
DCAMA[ECCV2022] [9]	41.5	46.2	45.2	41.3	43.5	48.0	58.0	54.3	47.1	51.9
IPMT[NIPS2022] [6]	40.5	45.7	44.8	39.3	42.6	45.1	50.3	49.3	46.8	47.9
PFENet[TPAMI2020] [14]	Res-50	34.3	33.0	32.3	30.1	32.4	38.5	38.6	38.2	34.3	37.4
RePRI[CVPR2021] [54]	32.0	38.7	32.7	33.1	34.1	39.3	45.4	39.7	41.8	41.6
HSNet[ECCV2021] [8]	36.3	43.1	38.7	38.7	39.2	43.3	51.3	48.2	45.0	46.9
CyCTR[NIPS2021] [7]	38.9	43.0	39.6	39.8	40.3	41.1	48.9	45.2	47.0	45.6
BAM[CVPR2022] [55]	43.4	50.6	47.5	43.4	46.2	49.3	54.2	51.6	49.6	51.2
IPMT[NIPS2022] [6]	41.4	45.1	45.6	40.0	43.0	43.5	49.7	48.7	47.9	47.5
DCAMA[ECCV2022] [9]	41.9	45.1	44.4	41.7	43.3	45.9	50.5	50.7	46.0	48.3
VAT[ECCV2022] [48]	39.0	43.8	42.6	39.7	41.3	44.1	51.1	50.2	46.1	47.9
HDMNet[CVPR2023] [13]	43.8	55.3	51.6	49.4	50.0	50.6	61.6	55.7	56.0	56.0
AMFomer (ours)	44.9	55.8	52.7	50.6	51.0	52.0	61.9	57.4	57.9	57.3
set it to 0.1 in experiments. Similarly,  $D$  is not optimized when training  $G$ . Following [13], we also include a KL (Kullback-Leibler) divergence loss between the correlation maps of adjacent stages to distill informative semantic cues of earlier stages to refine the segmentation quality. Please refer to [13] for more details. The overall loss function for  $G$  is:
$$
\mathcal {L} _ {g} = - \log \left(\mathbf {F C} \left(\boldsymbol {\omega} _ {f} ^ {k}\right)\right) + \mathcal {L} _ {K L} + B C E \left(\mathbf {M} _ {e}, \mathbf {M} _ {g t}\right). \tag {14}
$$
Where the  $BCE$  is the Cross-entropy loss. After training, only the well-learned  $G$  is needed in the inference stage.
K-shot inference. When  $K$  support samples  $\{I_s^k, M_s^k\}_{k=1}^K$  are available, different discriminative target regions  $\{\mathbf{M}_{q\tau}^i\}_{i=1}^K$  can be obtained from different support images via Eqn (4) and Eqn (5). We adopt the union of all the activated regions as the initial object region, i.e.,  $\mathbf{M}_{q\tau} = \mathbf{M}_{q\tau}^1 \cap \mathbf{M}_{q\tau}^2 \cap \dots \cap \mathbf{M}_{q\tau}^K$ . The comprehensive  $\mathbf{M}_{q\tau}$  originating from multiple supports is more robust to pose differences, occlusion, etc., than that from a single support image, thereby boosting the performance.
We evaluate the proposed AMFormer on two commonly used few-shot segmentation benchmarks, Pascal-5 $^i$  [2] and COCO-20 $^i$  [56]. Pascal-5 $^i$  is constructed based on the PASCAL VOC 2012
Table 4: Comparison of FB-IoU and the number Table 5: Ablation studies for different compo- of learnable parameters on COCO-20  ${}^{i}$  . nents and architecture design in AMFormer.
Backbone	Methods	FB-IoU (%)	#learnable
params	OM	DM	mIoU	Δ
1-shot	5-shot	Single	Multi	w/o L_div	w/ L_div
Res-50	HSNet [8]	60.4	67.0	10.4M	65.0	0.0
BAM [55]	68.2	70.7	2.6M	✓	66.5	+1.5
DCAMA [9]	69.5	71.7	47.7M	✓	69.5	+4.5
HDMNet [6]	72.2	77.7	4.2M	✓	✓	69.7	+4.7
AMFormer (ours)	72.9	78.8	5.1M	✓	✓	70.7	+5.7
Table 6: Performance comparison on  $1^{st}$  split of varying the number of local proxies.
#Part	4	6	8	10	12	14
mIoU	70.7	71.0	71.2	71.3	71.2	71.2
Table 7: mIoU of different kinds of labels (1st split).
mask	bbox	scribbles
71.3	70.5	70.0
Table 8: Quantitative measurement of intra- and interobject similarity.
Intra-object	Inter-object
0.015	0.546
dataset [57] and additional annotations from SBD [58]. Following previous works [14, 55], we equally divide the 20 categories into four splits, three of which for training and the rest one for testing. COCO-  $20^{i}$  is a larger benchmark based on MSCOCO dataset [59], the 80 categories of which are partitioned into four splits for cross-validation as down in [14]. We randomly sampled 1000 episodes from the testing split for evaluation. Following common precepts [37, 14, 55, 38], we adopt mean intersection-over-union (mIoU) and foreground-background intersection-over-union (FBIoU) as evaluation metrics.
We adopt ResNet-50 and ResNet-101 [60] as the backbone network in our experiment. Following previous works [14, 7, 13], we concatenate the features from the  $3^{rd}$  and  $4^{th}$  blocks of backbone and exploit a  $1 \times 1$  convolution layer to generate  $\mathbf{F}_s$  and  $\mathbf{F}_q$  of middle-level to avoid overfitting. The number of attention layers in the  $G$  and  $D$  are set to 1 and 2, respectively. We employ the same data augmentation setting as [14]. Since the  $G$  and  $D$  of AMFormer are trained alternately, we increase the number of training epochs to 300 for Pascal- $5^i$  and 75 for COCO- $20^i$ , and set the batch sizes as 8 and 4, respectively. AdamW [61] optimizer with poly learning rate decay is used to train both the  $G$  and  $D$ . The initial learning rate is set to  $1e^{-4}$  and the weight decay is  $1e^{-2}$ . It should be noted that we adopt a base learner to filter the categories that appear during the training process for a fair comparison with previous works [55, 13]. For more details please refer to the Supplementary Material. Our approach is implemented using PyTorch and all experiments are conducted on 4 NVIDIA GeForce RTX 3090 GPUs.
We present the comparison of our method with previous FSS methods on Pascal-  $5^{i}$  and COCO-  $20^{i}$  datasets in Table 2 and Table 3. It can be observed that the proposed AMFormer significantly outperforms previous advanced approaches and achieves new state-of-the-art results under all settings. Specifically, on Pascal-  $5^{i}$ , our AMFormer achieves  $70.4\%$  and  $73.2\%$  mIoU when using ResNet-50 as the backbone for 1-shot and 5-shot settings, surpassing the most competitive HDMNet [13] by  $1.0\%$  and  $1.4\%$ , respectively. With ResNet-101 backbone, our method outperforms the previous best results [6] by  $2.8\%$  (1-shot) and  $1.6\%$  (5-shot). We can obtain additional improvement when using larger backbone network, which demonstrates the scalability of the AMFormer. We attribute this performance gain to better intra-object similarity within the more informative features. As for the more complicated COCO-  $20^{i}$ , our approach also exhibits superior performances compared to other methods, demonstrating its competitiveness on complex data. Besides, Table 4 gives the comparison with previous methods in terms of FBIoU on Pascal-  $5^{i}$  using ResNet-50 backbone. AMFormer also outperforms all of the previous works by a considerable margin. Qualitative results are shown in Figure 4, please refer to Supplementary Material for analysis.
Figure 4: Qualitative results from different stages.
Figure 5: Visualization of regions activated by different local proxies.
As shown in Table 5, a series of ablation studies are conducted on the first split of Pascal-  $5^{i}$  with ResNet-101 backbone to analyze each component of the proposed AMFormer. Note that the first line of Table 5 is the result of our ablation baseline. The baseline adopts self-attention within the query features for feature parsing, and cross-attention to aggregate support information into query features as done in [7].
Effectiveness of object mining transformer  $G$ . We first construct a naive single-scale  $G$  as the  $2^{n}d$  row of Table 5. We can observe a significant performance lift, i.e.,  $1.5\%$  in mIoU. This improvement demonstrates the effectiveness of the proposed query-centric strategy, which is built upon intra-object similarity and thus is less affected by intra-class diversity. The multi-scale implementation of  $G$  further brings a  $3.4\%$  improvement in mIoU, and it already achieves a decent performance. It shows the importance of multi-scale feature aggregation in dealing with objects with dramatically changing scales in the query image.
Effectiveness of detail mining transformer  $D$ . The comparison between the  $3^{rd}$  and the  $5^{th}$  row of Table 5 shows that the combination of  $D$  improves the performance by  $1.2\%$  mIoU on the basis of  $G$ . We attribute this performance gain to the exploration and distinction of local features in the  $D$ , which encourages  $G$  to pay more attention to these ambiguous regions, e.g., object boundaries. Compared with the pixel-level supervision given by Cross-entropy loss, the proposed part-level adversarial training can incorporate local region context information to guide accurate segmentation. Note that without  $\mathcal{L}_{div}$ , the performance improvement brought by  $D$  drops to  $0.2\%$ . This phenomenon is reasonable because in the absence of  $\mathcal{L}_{div}$ , local proxies tend to degrade to focus on the same region, leaving some details unexplored.
Investigation of the local proxies. We first visualize the region activated by the learnable proxies to analyze what they mainly focus on. As shown in Figure 5, we observe that the highlighted area mainly lies on the boundary of the target and different proxies correspond to different directions. It confirms that the local proxies can well capture the local misalignment of the relatively coarse prediction from  $G$ , and then  $G$  is optimized to pay more attention to these ambiguous areas under the direction of  $D$ . In addition, we investigate the impact of the number of proxies on performance. As reported in Table 6, using more proxies can gradually improve the performance until the number achieves 10. More proxies no longer bring additional improvement. This result is expected, as too many proxies would learn redundant patterns.
Discussion on the query-centric FSS paradigm. Since the FSS models are trained in the extremely low data regime
Figure 6: Visualization of the activation of query images..
with the backbone not optimized, the feature space is not well aligned for novel classes. In this situation, the novel class data distribution usually has low coverage in the feature space, i.e., object features of the same category but different image instances may be far apart. On the contrary, as we can observe in Table 8, pixel features within the same object are closer to each other than those across objects. In a nutshell, intra-object similarity is more reliable than intra-category but inter-object similarity. As shown in Figure 6, we visualize the attention weight of query features between support target and pseudo support, respectively. It shows that support targets tend to mistakenly activate background categories, while the pseudo support can well excavate the full object attribute to intra-object similarity.
Owing to the reduced reliance on support information, query-centric FSS methods could achieve remarkable performance with more weak support annotations. Table 7 shows the results of the proposed AMFormer on the first split of Pascal-  $5^{i}$  with bounding boxes or scribbles as support labels. The promising results demonstrate the feasibility of more general FSS segmentation models.
4.5 Broader Impact and Limitations.
We proposed a novel query-centric FSS paradigm that shifts the research focus from support to query features. This is a new perspective that may inspire the development of more general FSS models that can be adopted in different tasks such as video object segmentation [62-64] or open-vocabulary segmentation [65, 66]. Although our AMFormer achieves remarkable performance, the number of training epochs is larger than some of previous approaches since the  $G$  and  $D$  in the AMFormer are trained alternately.
In this paper, we propose a novel query-centric FSS method, i.e., Adversarial Mining Transformer (AMFormer), which can achieve accurate query segmentation with only rough support guidance. Extensive experimental results demonstrate the superiority of our method. The decent performance with weak support labels also demonstrates the potential of the query-centric FSS paradigm.
This work was partially supported by the National Defense Basic Scientific Research Program of China (Grant JCKY2021601B013).
[1] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3431-3440, 2015.
[2] Amirreza Shaban, Shray Bansal, Zhen Liu, Irfan Essa, and Byron Boots. One-shot learning for semantic segmentation. arXiv preprint arXiv:1709.03410, 2017.
[3] Gen Li, Varun Jampani, Laura Sevilla-Lara, Deqing Sun, Jonghyun Kim, and Joongkyu Kim. Adaptive prototype learning and allocation for few-shot segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8334-8343, 2021.
[4] Bingfeng Zhang, Jimin Xiao, and Terry Qin. Self-guided and cross-guided learning for few-shot segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8312-8321, 2021.
[5] Jie Liu, Yanqi Bao, Guo-Sen Xie, Huan Xiong, Jan-Jakob Sonke, and Efstratios Gavves. Dynamic prototype convolution network for few-shot semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11553-11562, 2022.
[6] Yuanwei Liu, Nian Liu, Xiwen Yao, and Junwei Han. Intermediate prototype mining transformer for few-shot semantic segmentation. arXiv preprint arXiv:2210.06780, 2022.
[54] Malik Boudiaf, Hoel Kervadec, Ziko Imtiaz Masud, Pablo Piantanida, Ismail Ben Ayed, and Jose Dolz. Few-shot segmentation without meta-learning: A good transductive inference is all you need? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13979-13988, 2021.
[55] Chunbo Lang, Gong Cheng, Binfei Tu, and Junwei Han. Learning what not to segment: A new perspective on few-shot segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8057-8067, 2022.
[56] Khoi Nguyen and Sinisa Todorovic. Feature weighting and boosting for few-shot segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 622-631, 2019.
[57] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The Pascal visual object classes (voc) challenge. International journal of computer vision, 88(2):303-338, 2010.
[58] Bharath Hariharan, Pablo Arbeláez, Ross Girshick, and Jitendra Malik. Simultaneous detection and segmentation. In European conference on computer vision, pages 297-312. Springer, 2014.
[59] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dólár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740-755. Springer, 2014.
[60] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[61] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019.
[62] Ho Kei Cheng, Yu-Wing Tai, and Chi-Keung Tang. Rethinking space-time networks with improved memory coverage for efficient video object segmentation. Advances in Neural Information Processing Systems, 34:11781-11794, 2021.
[63] Rui Sun, Yuan Wang, Huayu Mai, Tianzhu Zhang, and Feng Wu. Alignment before aggregation: trajectory memory retrieval network for video object segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1218-1228, 2023.
[64] Rui Sun, Naisong Luo, Yuan Wang, Yuwen Pan, Huayu Mai, Zhe Zhang, and Tianzhu Zhang. 1st place solution for youtuves challenge 2022: Video object segmentation. In Proceedings of the Conference on Computer Vision and Pattern Recognition Workshops, 2022.
[65] Hang Zhao, Xavier Puig, Bolei Zhou, Sanja Fidler, and Antonio Torralba. Open vocabulary scene parsing. In Proceedings of the IEEE International Conference on Computer Vision, pages 2002–2010, 2017.
[66] Mengde Xu, Zheng Zhang, Fangyun Wei, Han Hu, and Xiang Bai. Side adapter network for open-vocabulary semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2945-2954, 2023.
7 More Details for Multi-scale Object Mining Transformer.
In the object mining transformer  $G$ , we alternately use down-sampling layers and self-attention layers to construct hierarchical query features. And the corresponding pseudo support features are obtained by the Hadamard product of the downsampled  $M_{q\tau}$ , specifically,
$$
\mathbf {F} _ {q, l} = \operatorname {D o w n} \left(\mathcal {F} ^ {- 1} \left(\operatorname {F e a t A g g} \left(\mathcal {F} \left(\mathbf {F} _ {q, l - 1}\right), \mathcal {F} \left(\mathbf {F} _ {q, l - 1}\right)\right)\right)\right), \tag {15}
$$
where the  $\mathcal{F}:\mathbb{R}^{H\times W\times C}\mapsto \mathbb{R}^{HW\times C}$  is the spatial flatten operation and  $\mathbf{Down}$  denotes the down-sampling layers, which is implemented with convolutional layers of double strides. In this way, we obtain multi-scale query features  $\mathbf{F}_{q,l}\in \mathbb{R}^{\frac{H}{2^{l - 1}}\times \frac{W}{2^{l - 1}}\times C},l = 1,\ldots ,L$  and pseudo support features  $\mathbf{F}_{psd,l} = \mathbf{F}_{q,l}\odot \varphi (M_{q\tau})$  . ( $\varphi$  is bilinear interpolation operation). We perform feature aggregation within each scale to enable contextual information exploration, thus avoiding the spatial inconsistency in the query image.  $L$  is set to 3 in our experiments. The embedding dim is set to 64, and the number of head is set to 4 in all the attention layers.
8 Detailed Experimental Settings
To achieve a fair comparison with previous methods [55, 13], we adopt the ensemble strategy following BAM [55] to filter the base categories seen during training. Specifically, a base learner is trained using the training splits in a fully supervised manner, and the learned base learner is used to explicitly predict the targets of base classes. PSPNet [18] is adopted as the base learner in all of our experiments, and we follow the BAM [55] to ensemble the prediction of the base learner and the proposed AMFormer. Our code is available at https://github.com/Wyxdm/AMNet
We also conduct an additional ablation experiment to evaluate the influence of the ensemble strategy as shown in Table 9. We can observe that the ensemble strategy can incrementally improve performance. It should be noted that our AMFormer can also surpass previous state-of-the-art methods (IPMT [6]) without the ensemble strategy.
Table 9: Ablation of ensemble strategy on Pascal-5 $^i$  with ResNet-101 backbone and 1-shot setting.
fold0	fold1	fold2	fold3	mean
IPMT [6]	71.6	73.5	58.0	61.2	66.1
w/o ensemble	69.7	75.2	69.5	62.9	69.3
w/ ensemble	71.3	76.7	70.7	63.9	70.7
9 Dataset Settings
We respectively divided Pascal-  $5^{i}$  and COCO-  $20^{i}$  into four splits following [14] for cross-validation. In Table 10 and Table 11, we provide the detailed split settings.
Table 10: Detailed splits setting of PASCAL-5
Fold	Test classes
PASCAL-50	aeroplane, bicycle, bird, boat, bottle
PASCAL-51	bus, car, cat, chair, cow
PASCAL-52	diningtable, dog, horse, motorbike, person
PASCAL-53	potted plant, sheep, sofa, train, tv/monitor
10 More Experimental Results
10.1 Quantitative analysis of intra-object similarity.
We compute the average pairwise pixel similarity from the same object (intra-object) and different objects from the support and query images of the same category (inter-object) using the cosine
Table 11: Detailed splits setting of COCO-20
Fold	Test classes
COCO-200	Person, Airplane, Boat, Park meter, Dog, Elephant, Backpack, Suitcase, Sports ball, Skateboard, W. glass, Spoon, Sandwich, Hot dog, Chair, D. table, Mouse, Microwave, Fridge, Scissors
COCO-201	Bicycle, Bus, T.light, Bench, Horse, Bear, Umbrella, Frisbee, Kite, Surfboard, Cup, Bowl, Orange, Pizza, Couch, Toilet, Remote, Oven, Book, Teddy
COCO-202	Car, Train, Fire H., Bird, Sheep, Zebra, Handbag, Skis, B. bat, T. racket, Fork, Banana, Broccoli, Donut, P. plant, TV, Keyboard, Toaster, Clock, Hairdrier
COCO-203	Motorcycle, Truck, Stop, Cat, Cow, Giraffe, Tie, Snowboard, B. glove, Bottle, Knife, Apple, Carrot, Cake, Bed, Laptop, Cellphone, Sink, Vase, Toothbrush
similarity. Note that the pixel features that we used to compute the similarity are middle-level features  $\mathbf{F}_s$  and  $\mathbf{F}_q$  as described in the L255-L256 in the original manuscript. The quantitative results of different categories are provided in Table 12 and Table 13. From the tables, we can observe that the intra-object similarity is at least one order of magnitude higher than the inter-object similarity. This demonstrates the superiority of the query-centric approach relying on intra-object similarity over support-centric methods that rely on inter-object similarity.