Anatomical Prior Guided Spatial Contrastive Learning for Few-Shot Medical Image Segmentation

Authors: wendong  huang
jinwu  hu
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-26 03:31:54

Abstract: Few-shot semantic segmentation has considerable potential for low-data scenarios, especially for medical images that require expert-level dense annotations. Existing few-shot medical image segmentation methods strive to deal with the task by means of prototype learning. However, this scheme relies on support prototypes to guide the segmentation of query images, ignoring the rich anatomical prior knowledge in medical images, which hinders effective feature enhancement for medical images. In this paper, we propose an anatomical prior guided spatial contrastive learning, called APSCL, which exploits anatomical prior knowledge derived from medical images to construct contrastive learning from a spatial perspective for few-shot medical image segmentation. The new framework forces the model to learn the features in line with the embedded anatomical representations. Besides, to fully exploit the guidance information of the support samples, we design a mutual guidance decoder to predict the label of each pixel in the query image. Furthermore, our APSCL can be trained end-to-end in the form of episodic training. Comprehensive experiments on three challenging medical image datasets, i.e., CHAOS-T2, MS-CMRSeg, and Synapse, prove that our method significantly surpasses state-of-the-art few-shot medical segmentation methods, with a mean improvement of 3.61%, 2.30%, and 6.38% on the Dice score, respectively.

Main paper: 
 introduction: 
 Automatic semantic segmentation on medical images is a vital prerequisite for various clinical trials and medical image analysis, such as treatment planning [1, 26] , radiation therapy [16, 21] , and disease diagnosis [11, 15] . When massive labeled data is available, segmentation models can achieve impressive results. However, in many clinical practices, segmentation models are typically susceptible to a severe lack of labeled data due to rare abnormal organ examples and the high cost of expert-level dense annotations.Recently, few-shot learning (FSL) [32] is presented as a promising solution to effectively address the above challenges, and few-shot learning based semantic segmentation methods are typically referred to as few-shot semantic segmentation (FSS) [24] . The key idea of FSS aims to employ class information extracted from labeled samples (called 𝑠𝑢𝑝𝑝𝑜𝑟𝑡) to instruct the segmentation of unlabeled images (called 𝑞𝑢𝑒𝑟𝑦). Accordingly, FSS typically learns transferable knowledge on multiple few-shot tasks (𝑒𝑝𝑖𝑠𝑜𝑑𝑒𝑠) from 𝑏𝑎𝑠𝑒 classes in the form of episodic learning, and episodically segment 𝑛𝑜𝑣𝑒𝑙 classes during inference.FSS methods have made significant advancements in the past few years, but the majority of them focus on segmentation tasks on natural images [2, 13, 14, 24, 33, 38] , while FSS methods tailored for medical images are quite rare [10, 23, 35] . This may be due to the difference between the intrinsic background of natural and medical images. Current FSS methods on natural images try to prevent the class representation from being mixed with semantic information from the background features, because complex and diverse background information tends to interfere with the segmentation results of the model. Differently, the distribution of organs in medical images (such as MRI or CT) is relatively stable. Other organs in the background features tend to contribute to locating and scoping the target organ. These background features should also be fully utilized for few-shot medical image segmentation, referred to as anatomical prior knowledge. Hence, FSS methods for natural images cannot be directly adopted to solve medical image segmentation tasks in few-shot scenarios. In the past few years, some works try to introduce FSL to solve segmentation problems of medical images in few-shot settings. Roy 𝑒𝑡 𝑎𝑙 . [23] design a prototypical learning framework based on squeeze and excitation blocks for multi-organ segmentation in CT scans via FSL. Kim 𝑒𝑡 𝑎𝑙 .[10] devise a bidirectional recurrent neural network based few-shot segmentation framework for volumetric medical image segmentation. Huang 𝑒𝑡 𝑎𝑙 . [6] generate adaptive prototype vectors from a vector quantization perspective and perform dense prediction by cosine similarity. Nevertheless, as depicted in Fig. 1 (a), the essential anatomical prior knowledge in medical images has not been fully considered and integrated into the existing FSS methods to solve the medical image segmentation problem in low-data regimes, which may lead to degradation of segmentation performance.To solve the aforementioned issue, we propose a novel anatomical prior guided spatial contrastive learning for medical image segmentation in few-shot scenarios, which fully exploits the limited annotations to improve the discriminability of feature representation, as depicted in Fig. 1 (b) . We refer to the proposed method as APSCL. To be specific, we incorporate anatomical information obtained from the prior distribution into the embedding features and then employ these features to construct spatial contrastive learning, to improve the distinguishability of the features learned by the model in data-scarce clinical medical scenarios. Furthermore, a novel mutual guidance decoder is proposed to leverage the guidance information in support samples to activate the co-occurrent features between support and query branches in an interactive guidance manner, which is beneficial to yield better segmentation results. Meanwhile, the spatial contrastive learning model combined with the new mutual guidance decoder enables our framework to be effectively trained end-to-end by the episodic learning paradigm for few-shot medical image segmentation. We evaluate our framework on three publicly available medical image segmentation datasets, 𝑖.𝑒., CHAOS-T2 [9] , MS-CMRSeg [42] , and Synapse [12] , in the 1-way 1-shot segmentation task.To recap, our contributions are as follows:• We propose to fully exploit anatomical prior knowledge inherent in both the support image and the query image to construct contrastive learning, from a spatial perspective, to facilitate the model to learn more distinguishable embedding features. To the best of our knowledge, this is the first time that anatomical prior knowledge is explicitly leveraged to solve a medical FSS problem via a probabilistic network. • We propose a mutual guidance decoder to yield the prediction mask for the query image, which can effectively recognize the query features derived by the model combined with spatial contrastive learning. • The proposed method significantly surpasses current stateof-the-art FSS methods and sets the new state-of-the-art FSS performance on three public medical image datasets. 
related work 2.1 few-shot semantic segmentation: 
 Few-shot learning (FSL) aims to learn transferable knowledge from limited annotated samples, which has been extensively used for image classification [32, 40] , object detection [3, 8] , and semantic segmentation tasks [2, 24] . For segmentation tasks, few-shot segmentation methods aim to achieve dense pixel prediction of unseen classes under the guidance of a small number of annotated samples. Shaban 𝑒𝑡 𝑎𝑙 . [24] first introduce FSL to solve segmentation tasks in low-data regimes and design a two-branch segmentation framework. Dong 𝑒𝑡 𝑎𝑙 . [2] design an episode-based prototypical segmentation network, which yields a representative prototype vector for each semantic category from few support samples and then uses these prototypes to instruct the segmentation of the query image. The prototypical paradigm is widely adopted in later research works [13, 33] and gradually applied to medical image segmentation tasks with extremely limited labeled data [7, 10, 23] . Roy 𝑒𝑡 𝑎𝑙 . [23] first introduce prototypical learning with squeeze and excitation mechanisms to solve the abdominal organ segmentation problem in volumetric CT data. Wu 𝑒𝑡 𝑎𝑙 . [35] employ semantic information from non-target slices to construct a multi-view contrastive learning strategy for medical image segmentation under few-shot scenarios.The recent studies mentioned above utilize support data to compute class prototypes and then use them to guide the segmentation of query images, which does not fully exploit the potential of the limited annotated data during training. In contrast, we propose to leverage anatomical prior knowledge in finite labeled data to construct contrastive learning, which contributes to boosting the discriminability of the learned features of the model.The latent diffusion model (LDM) [22] is a probabilistic model that learns a data distribution by progressively denoising a variable derived from a normal distribution, which is equivalent to learning the inverse procedure of a specific Markov chain. Constraining the variance between the approximate probabilistic distribution and the normal distribution is often regarded as a regularization technique, which is used for image segmentation tasks in cross-modal scenarios [36, 37, 39] . In this way, the network can learn a shared latent embedding space with the domain-invariant characteristic from input data through regularization. In this work, we further employ regularization to constrain the distribution difference between the latent embeddings of the input sample and the corresponding label, so that the network captures the intrinsic anatomical prior knowledge of medical images and facilitates the construction of contrastive learning.Contrastive learning is aimed at learning low-dimensional embedding representations of input samples by optimizing contrastive losses, which estimate the distances of various samples in the feature space. In the past few years, methods based on contrastive learning have proliferated and obtained state-of-the-art performance in common computer vision tasks. Accordingly, contrastive learning is gradually explored to solve few-shot learning problems [17, 18] . InfoPatch [17] improves contrastive learning by employing various data augmentations and the patch-wise relationship. Ouali 𝑒𝑡 𝑎𝑙 .[18] devise a spatial contrastive loss to learn local discriminative features to help few-shot classification. In contrast, there are currently only a few contrastive learning based methods for FSS tasks. Wu 𝑒𝑡 𝑎𝑙 . [35] construct dual contrastive learning to solve fewshot medical image segmentation tasks by exploiting information in non-target slices. By combining intrinsic properties of medical images with contrastive learning, we devise an anatomical prior guided spatial contrastive learning to solve the problem of medical image segmentation in the few-shot setting. 
methodology: 
 In this section, we first provide a detailed problem formulation for few-shot semantic segmentation (FSS). We then detail the proposed anatomical prior guided spatial contrastive learning (APSCL) and mutual guidance decoder (MGD). Finally, we present a novel training objective that can be optimized end-to-end.Few-shot semantic segmentation aims to train a model using a labeled training dataset 𝐷 𝑡𝑟𝑎𝑖𝑛 that can generalize well to a testing dataset 𝐷 𝑡𝑒𝑠𝑡 with only a handful of labeled images from unseen semantic classes, without retraining the model. In the FSS setting, training semantic classes 𝐶 𝑡𝑟𝑎𝑖𝑛 in 𝐷 𝑡𝑟𝑎𝑖𝑛 and unseen testing semantic classes 𝐶 𝑡𝑒𝑠𝑡 in 𝐷 𝑡𝑒𝑠𝑡 are disjoint, namely, 𝐶 𝑡𝑟𝑎𝑖𝑛 ∩𝐶 𝑡𝑒𝑠𝑡 = ∅. Assume 𝑁 represents the total number of semantic classes in 𝐶 𝑡𝑒𝑠𝑡 and 𝐾 represents the number of samples corresponding to each semantic class in 𝐶 𝑡𝑒𝑠𝑡 . The FSS problem is also known as the 𝑁 -way 𝐾-shot segmentation sub-problem.To quickly adapt to new classes and achieve segmentation in testing time, we adopt the episodic training for FSS, which is extensively employed in prior works [19, 35] . To simulate few-shot scenarios in the testing phase where each new class contains only 𝐾 labeled samples, the episodic training paradigm randomly samples a series of episodes from 𝐷 𝑡𝑟𝑎𝑖𝑛 for meta-training. Each episode is comprised of a support set 𝑆 𝑡𝑟𝑎𝑖𝑛 and a query set 𝑄 𝑡𝑟𝑎𝑖𝑛 . The support set 𝑆 𝑡𝑟𝑎𝑖𝑛 = {(𝑥 𝑖 𝑠 , 𝑦 𝑖 𝑠 (𝑐 𝑗 ))} consists of the images 𝑥 𝑖 𝑠 and the corresponding ground-truth masks 𝑦 𝑖 𝑠 (𝑐 𝑗 ), and is regarded as the reference of the current class 𝑐 𝑗 to be segmented; the query set 𝑄 𝑡𝑟𝑎𝑖𝑛 = {𝑥 𝑞 } consists of unlabeled images to be segmented. Here, the superscript 𝑖 = 1, 2, 3, ..., 𝐾 refers to the 𝑖-th image-mask pair of in 𝑆 𝑡𝑟𝑎𝑖𝑛 , and 𝑗 = 1, 2, 3, ..., 𝑁 refers to the number index of class 𝑐 ∈ 𝐶 𝑡𝑟𝑎𝑖𝑛 . During testing time, 𝐷 𝑡𝑒𝑠𝑡 is designed in the same paradigm, but where testing images and labels are of unseen semantic classes 𝐶 𝑡𝑒𝑠𝑡 . Notice that the background class marked as 𝑐 0 is not included in 𝐶 𝑡𝑟𝑎𝑖𝑛 and 𝐶 𝑡𝑒𝑠𝑡 .Since medical images are inherently low in contrast visibility, the boundary between the background tissues and the target organ is relatively blurred, which makes it challenging for the FSS model to yield desirable segmentation results on target organs. Besides, in low-data regimes, there may be extreme discrepancies in morphology and scale between support and query objects from the same category, called spatial inconsistency, which further undermines the performance of FSS methods. Hence, we explore the rich anatomical prior knowledge in medical images and from the perspective of spatial information to learn discriminative features that can well distinguish the foreground target from the background tissues. Accordingly, we construct anatomical prior guided spatial contrastive learning (APSCL) with support and query features, as illustrated in Fig. 2 . Anatomical prior generation with distribution consistency. Inspired by the distribution regularization in LDM [22] , we seek to reduce the distribution discrepancy between the latent embedding space of the input sample and the corresponding label from the perspective of regularization. Specifically, we first employ a feature encoder and a label encoder to map the input samples and labels to latent embedding variables, respectively. Afterwards, the Figure 2 : Workflow of our APSCL under the 1-way 1-shot scenario. The proposed method mainly consists of feature enrichment module (FEM), spatial contrastive learning (SCL), and mutual guidance decoder (MGD). The FEM is first proposed to enrich support and query features extracted by a CNN backbone with anatomical prior knowledge learned using distribution consistency. Then, the SCL is designed to exploit enriched features to construct contrastive learning from a spatial perspective, which acts as an auxiliary branch to facilitate the segmentation task. Finally, the MGD is introduced to interactively leverage the guidance information of support samples to yield the query segmentation result.approximate posterior distribution can be inferred in a parametric variational manner. The anatomical structure knowledge to be segmented, referred to as anatomical prior, can be captured by regularizing the discrepancy between the distribution of the input sample and its label.In our proposed network, a feature encoder 𝑓 𝜃 (•) is employed to embed the input samples 𝑥 into the feature space. The embedding representation is composed of a set of feature maps that are rich in abundant spatial information (𝑖.𝑒., spatial prior) of input sample 𝑥 in multiple channels. Given an input sample 𝑥, the corresponding embedding representation in the embedding space Λ is represented as 𝑓 𝜃 (𝑥). Since the latent embedding space Ω of the annotated ground-truth mask is mainly composed of the anatomical structure knowledge (𝑖.𝑒., anatomical posterior) of organs, we express the anatomical embedding representation of ground-truth mask 𝑦 as 𝑓 𝜙 (𝑦), where 𝑓 𝜙 (•) is a label encoder. Afterwards, 𝑓 𝜃 (𝑥) and 𝑓 𝜙 (𝑦) from the feature and label encoders are further fed into prior and posterior blocks with one flattening layer and three sequential linear layers, where these embedding representations are exploited to initialize the probability distributions of the latent spaces. In the latent representation space, the latent factor corresponding to the input sample is denoted as 𝑧 𝑥 , and the latent factor corresponding to the ground-truth mask is denoted as 𝑧 𝑦 . The approximate posterior probability distributions of 𝑧 𝑥 and 𝑧 𝑦 are denoted as 𝑃 (𝑧 𝑥 |𝑥, Λ) and 𝑃 (𝑧 𝑦 |𝑦, Ω), separately. In this way, the discrepancy between 𝑃 (𝑧 𝑥 |𝑥, Λ) and 𝑃 (𝑧 𝑦 |𝑦, Ω) can be considered as an effective regularization term for spatial contrastive learning and few-shot segmentation, which is formulated as:EQUATIONwhere D 𝐾𝐿 is Kullback-Leibler (KL) divergence. L 𝐾𝐿 gradually converges during the training process. Since in the meta-training process, we employ a backbone with shared weights to extract features from the support and query samples, the above anatomical prior generation can be applied in parallel to the support and query branches. Similarly, the label encoder also adopts such a parallel manner to encode support and query labels and has the same structure as the feature encoder. It is worth noting that in the testing phase, the generation of latent factors only relies on the input support and query images. While during the training process, we exploit the input support and query images and the corresponding labels to learn the posterior distribution. As shown in Fig. 2 , L 𝑠 𝐾𝐿 and L 𝑞 𝐾𝐿 represent the KL divergence on the support branch and query branch, respectively.Feature enrichment with anatomical prior. To better guide the activation of anatomical targets, motivated by FiLM [20] , we devise the feature enrichment module (FEM) with anatomical prior. The module aims to perform the affine transformation on the embedded features according to the data sampled from the anatomical approximate posterior probability distribution to achieve feature enhancement. The rescaling and translation factors used for the affine transformation are predicted with the latent factor 𝑧 𝑥 . To be specific, 𝑧 𝑥 is first sampled from latent embedding space, and then fed separately into two 3-layer MLPs with different parameter initializations to yield the scale 𝛽 and the shift 𝛼. For convenience, 𝛽 and 𝛼 are expanded to the same spatial resolution as the embedding feature 𝑓 (𝑥). Afterwards, the affine-transformed features are passed to a convolutional layer to obtain enhanced features with anatomical prior, and this procedure is expressed as follows:EQUATIONwhere the subscript 𝑡 denotes the feature map from the 𝑡-th channel in 𝑓 (𝑥). Likewise, the above feature enrichment process based on anatomical prior can be applied to both support and query branches to produce new support and query features. Spatial interaction module. In few-shot settings, segmentation models are prone to suffer from the spatial inconsistency problem between supports and query targets, and most FSS methods exploit multi-level spatial information to alleviate this problem. However, since multi-level spatial information cannot fundamentally affect the feature space, which tends to only achieve suboptimal solutions. For this purpose, we present a novel spatial contrastive learning strategy to obtain a feature space sensitive enough to spatial information, which optimizes the model via the spatial similarity between pairs of given support and query samples. To effectively activate spatial information, inspired by the Transformer [31] , we design a spatial interaction module (Fig. 3 ), which can compare support and query features from the perspective of spatial information and construct a spatial contrastive loss. Taking Fig. 3 (a) as an example, assume 𝑔 𝑞 , 𝑔 𝑘 and 𝑔 𝑣 denote the query, key, and value encoders, which take the enhanced features 𝑓 ′ (𝑥) as input and then output the query Q, key K and value V, respectively:EQUATIONwhere 𝑔 𝑞 , 𝑔 𝑘 and 𝑔 𝑣 are implemented as 1×1 convolutions, and the embedding dimension of Q, K, V ∈ R 𝐻𝑊 ×𝐸 ′ is denoted by 𝐸 ′ . Given a support-query feature pair (𝑖.𝑒., 𝑓 ′ (𝑥 𝑠 ) and 𝑓 ′ (𝑥 𝑞 )), SIM aims to obtain spatially aligned values of 𝑓 ′ (𝑥 𝑠 ) relative to 𝑓 ′ (𝑥 𝑞 ), regarded as V 𝑠 |𝑞 . This spatial alignment process consists of two steps. It first obtains the spatial attention weight 𝑊 𝑠,𝑞 ∈ R 𝐻𝑊 ×𝐻𝑊 by calculating the affinity matrix between the query Q 𝑠 of 𝑓 ′ (𝑥 𝑠 ) and the key K 𝑞 of 𝑓 ′ (𝑥 𝑞 )). Afterwards, 𝑊 𝑠,𝑞 is employed to weight V 𝑠 to yield V 𝑠 |𝑞 . The process can be mathematically defined as:EQUATIONEQUATIONLikewise, by swapping the roles of support and query samples, we can also obtain the alignment value V 𝑞 |𝑠 of 𝑓 ′ (𝑥 𝑞 ) relative to 𝑓 ′ (𝑥 𝑠 ).Spatial contrastive learning. With the alignment values V 𝑠 |𝑞 and V 𝑞 |𝑠 for the support and query features, we first perform 𝑙 2 normalization on the value V 𝑙 of each spatial location 𝑙, where V 𝑙 indicates the feature vector with embedding dimension 𝐸 ′ . Afterwards, the spatial relevance 𝑟 (𝑓 ′ (𝑥 𝑠 ), 𝑓 ′ (𝑥 𝑞 )) between a support-query feature pair can be written as:EQUATIONBased on the spatial relevance 𝑟 , we construct a novel spatial contrastive learning:EQUATIONEQUATION)where 𝜏 is the temperature coefficient and I ∈ {0, 1} is an indicative function.To obtain segmentation results, existing methods directly utilize support samples to guide the final segmentation of the query image. Nevertheless, such methods ignore the reference value of support images relative to query images. Therefore, we devise a mutual guidance decoder (MGD) to enhance query features via information interaction and yield more accurate segmentation results. To be specific, for the features of input support and query 𝑓 ′ (𝑥 𝑠 ), 𝑓 ′ (𝑥 𝑞 ) ∈ R 𝐻 ×𝑊 ×𝐸 , we first transform them into two embedding spaces ℎ 𝑠 , ℎ 𝑞 , where 𝑍 𝑠 = ℎ 𝑠 (𝑓 ′ (𝑥 𝑠 )), 𝑍 𝑞 = ℎ 𝑞 (𝑓 ′ (𝑥 𝑞 )). And ℎ 𝑠 and ℎ 𝑞 are implemented as 1×1 convolutions. Then, these features are reshaped to Ẑ𝑠 , Ẑ𝑞 ∈ R 𝑀 ×𝐸 1 . Multiple matrix multiplications are performed to yield self-attention and cross-attention respectively, as depicted in Fig. 4 . These two attentions are aggregated together via pixel-wise multiplication operations, which is formulated as:EQUATIONwhere the superscript T refers to the matrix transpose operation and 𝛼 denotes the aggregated attention along the channel dimension. Afterwards, the aggregated attention is employed to reweight the query feature Ẑ𝑞 , mathematically defined as:EQUATION)where Z𝑞 denotes the enhanced query features and the co-occurrent features are effectively highlighted in the support and query branches.Following [41] , we take a non-parametric way to calculate the class prototypes, namely, masked average pooling. Given the support features and corresponding masks, the prototype P 𝑐 of class 𝑐 is computed as follows:EQUATIONwhere (𝑥, 𝑦) represents the coordinates in the spatial dimension and 𝐾 refers to the quantity of support samples belonging to class 𝑐. Accordingly, with the augmented query feature Z𝑞 and support prototypes P ={𝑝 𝑐 |𝑐∈𝐶}, we compute the cosine similarity between them to obtain the final query prediction mask ŷ𝑞 :EQUATIONwhere softmax(•) denotes the normalized exponential function.In the proposed framework, we employ cross-entropy L 𝐶𝐸 to evaluate the segmentation loss of the query image. Kullback-Leibler divergence L 𝐾𝐿 and spatial contrastive loss L 𝑆𝐶𝐿 are employed as auxiliary losses to facilitate the model to learn embedding features with sufficient discriminability. Therefore, the total objective of APSCL is expressed as:EQUATION)where the 𝜆 𝐾𝐿 and 𝜆 𝑆𝐶𝐿 are the trade-off weights, and are experimentally specified as 0.1 and 0.07, respectively. 
experiments 4.1 experimental setting: 
 Datasets. We examine the performance of our APSCL on three publicly available medical benchmark datasets, namely, CHAOS-T2 [9] , MS-CMRSeg [42] , and Synapse [12] : (1) CHAOS-T2 is derived from the 2019 ISBI Combined Healthy Abdominal Organ Segmentation Challenge, which consists of 20 3D abdominal MRI scans with a total of 623 axial 2D slices per scan. (2) MS-CMRSeg is from the 2019 MICCAI Multi-sequence Cardiac MRI Segmentation Challenge, which includes 35 3D cardiac MRI scans with a mean of 13 2D slices per scan. (3) Synapse stems from the 2015 MICCAI Multi-Atlas Abdomen Labeling Challenge, which is comprised of 30 3D organ CT scans with a total of 3779 axial 2D slices per scan.To simulate the paucity of annotated samples in clinical settings, we carry out all experiments in the 1-way 1-shot setting. In the CHAOS-T2 and Synapse scans, we segment four classes: liver, left kidney (LK), right kidney (RK), and spleen. In the MS-CMRSeg scans, we segment three organ classes: left-ventricle blood pool (LV-BP), right-ventricle (RV), and left-ventricle myocardium (LV-MYO). We choose one of the organ classes as the unseen testing class (𝑖.𝑒., 𝑛𝑜𝑣𝑒𝑙 class) and the others as the seen training classes (𝑖.𝑒., 𝑏𝑎𝑠𝑒 classes) to form multiple few-shot tasks. In all experiments, the five-fold cross-validation is employed to evaluate the models.Implementation details. The proposed APSCL is implemented by PyTorch on an NVIDIA Tesla V100 GPU with 32 GB memory. A popular ResNet-101 [5] backbone is employed as the encoder 𝑓 (•), which is pre-trained on the MS-COCO dataset [27] following the common practice [19] . It processes 3 × 256 × 256 dimensional input data and generates 256 × 32 × 32 dimensional embedding features. During anatomical prior generation, the number of latent factors is experimentally specified as 32. During training, the entire learning process is optimized by the stochastic gradient descent optimizer with a learning rate of 1𝑒-3 and a batch size of 1 in an end-to-end manner. The weight decay and momentum are specified as 5𝑒-4 and 0.9, respectively. The whole network is trained for 100 epochs, each containing 1000 episodes. The Dice score is adopted as the performance metric in the experiments.In this section, we compare the proposed anatomical prior guided spatial contrastive learning (APSCL) and state-of-the-art FSS methods, including the baseline method SE-Net [23] , PANet [33] , SSL-ALPNet [19] , PoissonSeg [25] , RP-Net [29] , GCN-DE [28] , SRNet [34] , AAS-DCL [35] , ADNet [4] , and LVQM [6] . All the models are reimplemented in the same setting for a fair comparison. Table 1 reports the 1-shot results with respect to the Dice score. It is observed that the proposed APSCL achieves excellent performance in the 1-shot scenario, exceeding the previous FSS method by a significant margin. For example, compared to the baseline method SE-Net, our APSCL achieves 35.98%, 42.95%, and 40.30% improvement in terms of the mean Dice score on CHAOS-T2, MS-CMRSeg, and Synapse, respectively. In comparison with the state-of-the-art method LVQM, our APSCL surpasses LVQM by an average of 3.61%, 2.30%, and 6.38% on CHAOS-T2, MS-CMRSeg, and Synapse, respectively. It is because our method fully exploits the anatomical information in medical slices to guide the model to learn discriminative features suitable for clinical practice via contrastive learning, and more efficiently exploits the information of support features to guide segmentation via mutual guidance decoder. These results on three datasets demonstrate that considering anatomical information in medical images and guidance information from support samples is crucial for improving medical image segmentation models in few-shot scenarios.To intuitively illustrate the superiority of the proposed APSCL, we also present some examples of visual segmentation results of our APSCL and other methods on the CHAOS-T2 and Synapse datasets in Fig. 5 . It can be observed that our proposed APSCL yields desirable segmentation masks for organs with various intensities, scales, and morphologies in complex backgrounds while other methods yield inferior segmentation results on most organs. Since the comparison methods do not take into account anatomical prior information from medical images, segmentation masks are typically incomplete and inaccurate. This further proves the effectiveness of our APSCL for medical image segmentation in low-data regimes.In this section, we present ablation study results under the 1-way 1-shot scenario on the CHAOS-T2 dataset. Effect of major components. As illustrated in Table 2 , we investigate the impact of the major components by incrementally applying our proposed components to the baseline model, 𝑖.𝑒., SE-Net. Firstly, integrating each component individually into the baseline model can achieve performance gains. In particular, the FEM and SCL can improve the mean Dice score by 12.67% and 17.16%, respectively. When applying the two components together to the baseline model, it is observed that the FEM & SCL can substantially enhance the mean Dice score to 78.93%, while the FEM & MGD and SCL & MGD slightly boost segmentation performance. Moreover, Table 2 : Ablative results (in Dice score %) of various components of the proposed method on the CHAOS-T2 dataset. The baseline method is SE-Net [23] , namely the APSCL without the FEM, SCL, and MGD modules. the mean Dice score is further boosted to 85.47% when the above three parts are jointly integrated into the baseline model. The above In addition, we also present visualization results of ablation experiments, as shown in Fig. 6 . It can be intuitively observed that, in comparison to the baseline method, the anatomical prior generated by distribution consistency can lead to better segmentation masks with the assistance of the FEM. And after integrating the three essential components together into the baseline method, the segmentation masks are closer to the ground-truth masks. These visualization results further verify the effectiveness of the proposed three components.Visualization of pixel features. We project the obtained pixel features into 2D space using t-SNE [30] . As shown in Fig. 7 , the learned pixel features by APSCL become more compact and well separated compared to the baseline (𝑖.𝑒., SE-Net). This proves that the proposed APSCL can yield better segmentation performance for the unseen classes.Impact of the hyper-parameters 𝜆 𝐾𝐿 and 𝜆 𝑆𝐶𝐿 . In this experiment, we examine the impact of the trade-off hyper-parameters 𝜆 𝐾𝐿 and 𝜆 𝑆𝐶𝐿 in the total objective function, which control the effect of KL divergence and spatial contrastive loss in Eq. (13) . The experiments are conducted on the CHAOS-T2 dataset in the 1-way 1-shot setting. Table 3 presents the performance of the different hyper-parameter values of 𝜆 𝐾𝐿 and 𝜆 𝑆𝐶𝐿 respectively. As the hyperparameter values increase, 𝜆 𝐾𝐿 and and 𝜆 𝑆𝐶𝐿 exhibit favorable and then unfavorable effect, while both reasonable values exhibit a better upper bound on performance. Our APSCL is robust enough to the selection of hyper-parameters, and it reaches the optimal Influence of the size of the latent factor 𝑧 𝑥 . We also investigated the influence of the size of the latent factor 𝑧 𝑥 , and the experimental results are illustrated in Table 4 . The latent factor is used for anatomical prior generation. The ablation experiments are carried out in the 1-way 1-shot case on the CHAOS-T2 dataset. From Table. 4, it is observed that as the size of the latent factor is close to 32, the growth of the mean Dice score slows down while the growth in computational complexity remains large. Considering computation efficiency and performance, we set the size of the latent factor 𝑧 𝑥 to 32 in all experiments. 
conclusion: 
 In this work, we propose a novel anatomical prior guided spatial contrastive learning scheme, named APSCL, to tackle the challenging medical image segmentation tasks in extremely low-data regimes. Unlike existing FSS methods, our framework fully exploits anatomical prior information from medical images to construct contrastive learning from a spatial perspective to boost the distinguishability of the learned features. Moreover, the mutual guidance decoder is designed to fully exploit the guidance information of support samples to yield more accurate segmentation results. Our extensive experiments on CHAOS-T2, MS-CMRSeg and Synapse datasets demonstrate that the proposed APSCL surpasses current state-of-the-art FSS methods by as much as 6.38% with respect to the Dice score. 
 
 Appendix:  


Decision: The paper presents an innovative approach called Anatomical Prior Guided Spatial Contrastive Learning (APSCL) for few-shot medical image segmentation. Reviewers recognized the novelty and effectiveness of integrating anatomical prior knowledge with spatial contrastive learning, resulting in substantial performance improvements across multiple datasets. To further enhance the paper, reviewers suggested providing more detailed explanations of the anatomical priors generation, improving visualizations of key components, comparing with additional recent works, and including failure case analysis. The authors addressed these concerns in the rebuttal, leading to a consensus for acceptance, with the recommendation to incorporate these suggested improvements for increased clarity and impact.

Reviews: review: paper_summary: paper_summary: The paper introduces a novel algorithm known as Anatomical Prior Guided Spatial Contrastive Learning (APSCL), which leverages anatomical prior knowledge to enhance feature representation for few-shot medical image segmentation. The APSCL algorithm uniquely incorporates anatomical priors by enforcing distribution consistency, allowing the model to perform affine transformations on embedded features based on data sampled from the anatomical approximate posterior probability distribution. This process significantly enhances feature alignment and representation. Additionally, the paper proposes the use of a Spatial Interaction Module that utilizes a transformer mechanism to facilitate long-range interactions between support and query images, enhancing the spatial contrastive learning process before moving into a mutual guidance decoder for segmentation.
strength_weakness: strength_weakness: Novelty:
1. Innovative Use of Anatomical Priors: The APSCL framework is pioneering in its integration of anatomical priors with spatial contrastive learning, making it a significant departure from traditional prototype-based approaches in few-shot segmentation. This integration allows for an advanced feature alignment and enhancement that is specifically tailored to the complex requirements of medical imaging.
2. Spatial Interaction via Transformers: The incorporation of a transformer-based Spatial Interaction Module not only facilitates the generation of interaction features but also crucially models significant spatial deformations and misalignments that are not adequately addressed by the prior-driven affine transformations. This capability significantly enhances the model's ability to handle the dynamic and varied nature of medical image segmentation, where traditional methods may fall short.

Experimental Performance:
1. Superior Performance Metrics: The method demonstrates a substantial improvement over existing state-of-the-art approaches on three challenging medical image datasets—achieving mean improvements of 3.61%, 2.30%, and 6.38% on the Dice score respectively. These results underscore the effectiveness of the proposed method in practical, real-world scenarios.
2. Comprehensive Validation: Through rigorous experiments, including ablation studies(Table 2), the paper effectively demonstrates the critical components and overall efficacy of the APSCL framework. Furthermore, the exploration of weakly-supervised learning scenarios adds another layer of robustness to the validation, showing the versatility and adaptability of the approach.
limitations: limitations: 1. Detailed Explanation of Anatomical Priors Generation: The generation of anatomical priors, particularly the optimization of equation (1), requires more detailed clarification. It would be beneficial to specify the parametric form of the posterior probability distributions used, whether they are modeled using Gaussians or other probabilistic forms. For enhanced clarity, especially for readers unfamiliar with this methodology, a more thorough explanation in the main text or an appendix would be advisable.

2. Enhanced Visualization: The paper would greatly benefit from the inclusion of visualizations for the anatomical priors, the resulting affine transformations, and the attention mechanisms within the Spatial Interaction Module and Mutual Guidance Decoder. Given that these elements are central to the paper’s contributions and distinguish it from prior work on anatomical prior-based few-shot segmentation, detailed visual representations could significantly enhance understanding and appreciation of the novel approaches.

3. Inclusion of Experimental Baselines and Failure Case Analysis: The absence of experimental comparisons with fully supervised segmentation models and an analysis of failure cases is a notable limitation. Understanding the performance ceiling with fully supervised approaches on the same datasets could provide valuable context for the few-shot results, highlighting the performance gap and potential areas for improvement. Additionally, an analysis of situations where the proposed method excels or falls short, particularly across different imaging types, organs, or modalities, would offer deeper insights into the method’s limitations and inform future research directions.

4. Comprehensive Literature Review: The paper's review of related literature appears incomplete, particularly concerning recent advancements in anatomical-prior-driven few-shot and medical image segmentation. Works such as "Learning what and where to segment: A new perspective on medical image few-shot segmentation, Medical Image Analysis 2023" and "A location-sensitive local prototype network for few-shot medical image segmentation, ISBI 2021", as well as other work on anatomical-prior-based techniques could be reviewed and discussed.
paper_summary: paper_summary: The paper titled "Anatomical Prior Guided Spatial Contrastive Learning for Few-Shot Medical Image Segmentation" introduces a novel approach to improve few-shot medical image segmentation, which iscalled APSCL (Anatomical Prior Guided Spatial Contrastive Learning). It leverages anatomical prior knowledge to guide spatial contrastive learning. This method enhances feature discriminability by aligning features with embedded anatomical representations. Additionally, the framework includes a mutual guidance decoder that utilizes support sample information to predict pixel labels in query images effectively. The APSCL model can be trained end-to-end using episodic training and demonstrates significant performance improvements over existing methods on three challenging medical image datasets: CHAOS-T2, MS-CMRSeg, and Synapse.
strength_weakness: strength_weakness: - **Innovative Approach**: The use of anatomical prior knowledge and spatial contrastive learning is novel for the specific challenges of medical image segmentation in low-data scenarios.
- **Comprehensive Experiments**: The method was tested on three challenging medical image datasets, showing substantial improvements over state-of-the-art methods, which validates the robustness and generalizability of the proposed framework.
- **Detailed Ablation Studies**: The paper includes thorough ablation studies that highlight the contributions of different components of the APSCL framework, strengthening the validity of their design choices.
- **Presentation**: Well-organized presentation.
limitations: limitations: 1. Spatial Contrastive Learning: From line 455-457, the spatial inconsistency problem is suggested. What is a concrete example of it? For the spatial features Q,K,V, W, what are they represented for? I am not sure about the reasons behind using attention mechanism and asking two spatial features to be the same.
2. Anatomical Posterior: How do you model the latent distribution? I would assume the latent variable is just 1-D vectors, though in figure2, it is illustrated as a 2D map (line 352). Also, How is the use of latent method related to the term "anatomy"? 
3. Ablation study: Is there an ablation experiment for the use of anatomical prior? Replacing the current latent method on spatial prior and anatomical posterior with other segmentation loss to find out if it is the anatomic factor,  rather than the network structure, that works here.
paper_summary: paper_summary: This paper tackles the challenge of accurately segmenting medical images when limited training data is available. It introduces an innovative method called Anatomical Prior guided Spatial Contrastive Learning (APSCL) that leverages anatomical knowledge and spatial relationships to boost segmentation performance in low-data scenarios.
strength_weakness: strength_weakness: APSCL is a novel framework incorporating anatomical priors and spatial contrastive learning, representing a great contribution to medical image analysis.

Compared to state-of-the-art methods, APSCL demonstrates significant improvements in segmentation accuracy across three challenging datasets: CHAOS-T2 (3.61% higher Dice score), MS-CMRSeg (2.30% higher), and Synapse (6.38% higher).


The end-to-end training approach simplifies the implementation process, potentially making APSCL more adaptable to various datasets and use cases.
limitations: limitations: Integrating anatomical priors and spatial contrastive learning may make APSCL complex to implement. Specifically, there are three major components: FEM, SCL, and MGD. Some of these designs probably could be simplified. For example, the ablation study in Table 2 shows that MGD has the weakest effect. How would it be different with simple pooling and some linear layer, instead of mutual attention and gated pooling? 


For effective SCL, the method's effectiveness heavily depends on the quality and accuracy of the available anatomical prior knowledge. Poor-quality priors could diminish performance gains. The paper uses an effective kl regularization term for spatial contrastive learning and few-shot segmentation. However, this consistency is conducted in low-dimensional space after feature compression. The potential weakness could be the loss of fine-grained morphology prior in such space. 

Model comparison: Table 1 shows the proposed method, which achieved remarkable results over the baseline methods. The latest work in Table 1 is from CVPR 2023, which is insufficient to represent the up-to-date progress. Some recent works have not been compared or at least acknowledged. For example:
Tianang Leng, Yiming Zhang, Kun Han, Xiaohui Xie; Self-Sampling Meta SAM: Enhancing Few-Shot Medical Image Segmentation With Meta-Learning. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024, pp. 7925-7935


It would be interesting to see the potential limitations in the conclusion and to inspire future work.
paper_summary: paper_summary: Few-shot semantic segmentation has considerable potential for lowdata scenarios, especially for medical images that require expertlevel dense annotations. Existing few-shot medical image segmentation methods strive to deal with the task by means of prototype learning, but ignored the rich anatomical prior knowledge in medical images, which hinders effective feature enhancement for medical images. In this paper, authors propose an anatomical prior guided spatial contrastive learning, called APSCL, which exploits anatomical prior knowledge derived from medical images to construct contrastive learning from a spatial perspective for few-shot medical image segmentation. Comprehensive experiments on three challenging medical image datasets, CHAOS-T2, MS-CMRSeg, and Synapse, prove that APSCL significantly surpasses state-of-the-art few-shot medical segmentation methods, with a mean improvement of 3.61%, 2.30%, and 6.38% on the Dice score, respectively.
strength_weakness: strength_weakness: 1.	This paper proposes to use the anatomical prior knowledge to guide spatial contrastive learning to help achieve better medical image segmentation results in FSS.
2.	The validation experiments  are extensive and enough.
3.	The paper content is rich in graphics and tables, with clear formulas listed,
limitations: limitations: 1. Is the dataset used in the experiment consistent with the dataset used in LVQM[1]? The first two datasets seem to be the same dataset according to the description? 
2. If the experimental dataset is consistent, why are the experimental results inconsistent in this paper and LVQM[1]?
3. The visualization results of the experiment in Figure 5 would be better if supplemented with MS-CMRSeg.
4. A more detailed explanation is needed for the calculation of LKL, such as how the Prior Block and Posterior Block used are implemented? No explanation found in the text.

[1] Shiqi Huang, Tingfa Xu, Ning Shen, Feng Mu, and Jianan Li.  Rethinking Few-Shot Medical Segmentation: A Vector Quantization View. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2023, 3072–3081.