Segment Anything Model Meets Semi-supervised Medical Image Segmentation: A Novel Perspective
Authors: Haifeng Zhao
Haiyang Li
Leilei Ma
Dengdi Sun
Venue: NeurIPS 2025 poster ()
Keywords: Semi-supervised Segmentation
Segment Anything Model
Knowledge Distillation
Primary area: applications

Abstract: The scarcity of annotated medical imaging data has driven significant progress in semi-supervised learning to alleviate reliance on expensive expert labeling. While foundational vision models such as the Segment Anything Model (SAM) exhibit robust generalization in generic segmentation tasks, their direct application to medical images often results in suboptimal performance. To address this challenge, in this work, we propose a novel fully SAM-based semi-supervised medical image segmentation framework and develop the corresponding knowledge distillation-based learning strategy. Specifically, we first employ an efficient SAM variant as the backbone network of the semi‑supervised framework and update the default prompt embedding of SAM to unleash its full potential. Then, we utilize an original SAM, which is rich in prior knowledge, as the teacher to optimize our efficient student SAM backbone through hierarchical knowledge distillation and a dynamic loss weighting strategy. Extensive experiments on various medical datasets demonstrate that our method outperforms state-of-the-art semi-supervised segmentation approaches. Especially, our model requires less than 10% of the parameter size of the original SAM, enabling substantially lower deployment and storage overhead in real-world clinical settings.

Segment Anything Model Meets Semi-supervised Medical Image Segmentation: A Novel Perspective
Haifeng Z2 Haiyang Li Lei-Lei Ma Dengdi Sun
Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, School of Computer Science and Technology. Anhui University. China Key Lab Intelligent Comp Signal Proc, Minist Edue, Anhui University, China School of Artificial Intelligence, Anhui University, China
Abstract
The scarcity of annotated medical imaging data has driven significant progress in semi-supervised learning to alleviate reliance on expensive expert labeling. While foundational vision models such as the Segment Anything Model (SAM) exhibit robust generalization in generic segmentation tasks, their direct application medical images often results in suboptimal performance. To address this challenge, in this work, we propose a novel fully SAM-based semi-supervised medical image segmentation framework and develop the corresponding knowledge distillation- based learning strategy. Specifically, we first employ an efficient SAM variant as the backbone network of the semi-supervised framework and update the default prompt embedding of SAM to unleash its full potential. Then, we utilize an original SAM, which is rich in prior knowledge, as the teacher to optimize our efficient student SAM backbone through hierarchical knowledge distillation and dynamic loss weighting strategy. Extensive experiments on various medical datasets demonstrate that our method outperforms state-of-the-art semi-supervised segmentation approaches. Especially, model requires less than 10% of the parameter size of the original SAM, enabling substantially lower deployment and storage overhead in real-world clinical settings.
1 Introduction
Medical image segmentation aims to delineate precise anatomical structures from imaging data and serves fundamental basis for clinical applications 2 Recently, as important foundational vision model for general image segmentation, the Segment Anything Model (SAM) K has been applied medical images, resulting in various medical SAM variants. Through fine-tuning on medical
datasets 4 5 6 and generating high-quality prompts 77] g 9 [0]. these variants have achieved considerable segmentation results. Despite significant progress in medical image segmentation, the cumbersome and costly manual annotation process still hinders its development. To address this challenge. cemi-supervised learning has gained traction as robust approach that enhances model performance and generalization by leveraging both limited labeled data and abundant unlabeled data. However, there are still some key issues regarding how effectively integrate the powerful capabilities of foundational models into semi uemi-supervised medical image segmentation (SSMIS).
First, existing SSMIS frameworks have yet to fully leverage the capabilities of SAM. Lately, researchers have conducted numerous explorations into the application of SAM in SSMIS. For instance, SemiSAM [1T] leverages coarse masks from the segmentation model derive prompt points, which are then fed into SAM to generate more accurate pseudo-labels for model optimization. SFR [12] introduces an improved approach by pre-training SAM on labeled data, allowing it to
Cooreeponddncce to: Lei-Lei Ma (xiaolcilci19900@mmall
and Dengdi Sun (sundengdi @ 163.com).
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Teacher
Seperised (5%) (ivee)
Spersiiedd tivlle)
Teicher
Sentvvapariid (MT)
Semsequurial (Uur))
Pr tha
SAM
efficten
P
0.4
original
x
FMA
Scovee (DMEC/leeeee
1o Dietance (9111D
efficient
-P'
SAM
Ps
(difeeenn Student
Student
Learning Strategy
Learning Strategy
(a)
(b)
(C)
(a) Mean Teacher for ESDE. (b) Two semi-supervised approaches based on ESDE (Mean Teacher and our approach, colored in orange and red) with 5% labeled data and 95% unlabeled data vs. fully supervised learning (colored in blue) using 5% and 100% labeled data on the LA dataset. Our approach achieves competitive performance, with an 88. 88.42% DSC, improving by 9.33% over the 5% labeled supervised setting. (c) Our proposed approach for ESDE.
Figure 1:
directly generate high-quality pseudo-labels. However, due to the large parameter size of SAM, these approaches do not fully adopt SAM for semi-supervised segmentation but instead typically deploy it as auxiliary component in SSMIS only for generating pseudo-labels as an additional source of supervision. This strategy inevitably restricts the potential of SAM. Moreover, such implementations are critically dependent on the prompts generated by the segmentation model, and inaccurate prompts can severely degrade the segmentation performance of SAM [13]. Therefore, the key challenge lies in maximizing the segmentation advantages of SAM while minimizing the impact of prompts, which is critical to advancing its application in SSMIS.
It is worth noting that overcome the substantial computational demands of SAM, the community has proposed various lightweight approaches to improve efficiency without compromising accuracy. These methods focus on training lightweight models entirely from scratch 04
15
16
and use
knowledge distillation with appropriate supervision for model training 17
18
19
20
21. The
success of efficient SAM variants inspires us to consider whether they can serve as backbone networks
in semi-supervised learning frameworks, replacing traditional ConvNet ConvNet-based backbones to fully leverage their capabilities. Moreover, when employing SAM as the segmentation model, updating its default prompt embedding can naturally enable high-quality segmentation results, while also mitigating the negative effects of inaccurate prompts. Therefore, we propose novel fully SAMe-based SSMIS framework that utilizes an Efficient SAM variant as the backbone with a Default Embedding (ESDE), which is promising to address the aforementioned challenges.
Second, commonly used learning strategies are incompatible with the fully SAM-based SSMIS framework. As shown in Figure D (a), we present a preliminary analysis by applying ESDE to a popular semi-supervised learning approach, Mean Teacher (MT) [22), where a teacher model is updated via exponential moving average (EMA) of student model. However, as shown in Figure [] (b), despite the availability of extensive unlabeled data, ESDE with MT achieves only a marginal improvement of less than 4% Dice Similarity Coefficient (DSC). We attribute this limitation the ViT-based architecture of SAM, which has weaker inductive biases compared to Conv Nets, making it less effective in general semi-supervised paradigms 23 Therefore, another key objective of this work is to develop a simple yet effective semi- muni--spervisee learning strategy for ESDE, in which the efficient SAM variant can better benefit from unlabeled data.
Based on the above analysis, as illustrated in Figure I (c), we propose a knowledge distillation- based learning strategy coupled with customized training pipeline for ESDE to advance SSMIS. Specifically, we utilize an original SAM to optimize the efficient SAM backbone through knowledge distillation. The customized training process divided into two steps: first, we apply parameter- efficient fine-tuning to the original SAM, progressively adapting its general segmentation capability to medical-specifie tasks. Notably, this approach maintains the robustness of the foundational model while improving the precise recognition of anatomical structures. Then, the fine-tuned SAM serves as the teacher model, where it provides comprehensive guidance the efficient student SAM through a hierarchical knowledge transfer mechanism operating on global contextual features and local boundary details. To address potential limitations, dynamic weighting strategy adjusts the distillation intensity during training, thereby mitigating the excessive influence of the frozen teacher. Through the phased training pipeline, the framework effectively utilizes unlabeled data while preserving the segmentation strength of SAM in medical scenarios.

---

Overall, in this work, we design fully SAM-basee SSMIS framework and develop the corresponding knowledge distillation-basee learning strategy. Our main contributions are summarized as follows:
To our knowledge, we are the first to fully use efficient SAM as the backbone network in semi-superrised learning framework. Meanwhile, we update the default prompt embedding of SAM to avoid the negative impact of inaccurate prompts. This innovative attempt harnesses its full power while reducing the complexity of deployment.
We propose novel semi-supervised learning strategy to address the limitations of ESDE in traditional settings. It constructs hicrarchical knowledge distillation pipeline transfer structural knowledge and boundary details, and designs a dynamic weighting mechanism that adaptively adjusts distillation intensity based on the student's 's evolving capabilities.
Comprehensive experiments across various medical segmentation tasks demonstrate that our proposed method achieves superior performance compared to other state-of-the-art semi-supervised methods.
2 Related Work
2.1 Semi-supervised Medical Image Segmentation
Since medical image annotation requires specialized expertise, incurs high costs, and is time- consuming, semi-supervised learning (SSL) [24 25 26 has emerged as an effective approach to tackle the challenge of insufficient labeled data medical image segmentation. Most existing semi-supervised segmentation methods are typically grouped into two categories: consistency regu- larization [22] and pseudo-labeling (27]. Consistency regularization aims to learn more robust and generalizable representations by maintaining consistent predictions for the same input under different perturbations [28 [29. Pseudo-labeling leverages the high-confidence predictions of the model on unlabeled data as temporary labels for supervised training [30, 31]. Unlike these methods, which are typically optimized using ConyNet-based backbones, we propose introducing an efficient SAM variant as the backbone network and designing novel fully SAM-based framework.
2.2
SAM Adaptation for Medical Images
The Segment Anything Model (SAM) [3], trained on the large-scale SA-1B dataset, exhibits re- markable zero-shot generalization for natural images [32]. However, its performance drops sharply when applied to medical images [33 B 35), especially for unseen anatomical structures or patholo- gies [36 37). To better adapt SAM for medical images, researchers are focusing generalizing SAM using automatic prompting techniques and different fine-tuning strategies. For instance, MaskSAM (38) provides accurate guidance to SAM through the design of learnable prompt gener- ator. Med-SA SSA (5] fine-tunes the prompted SAM with points and boxes, incorporating lightweight adaptation blocks to extract domain-specific medical prior knowledge. Moreover, prompt-free SAM adaptation approaches introduced for medical segmentation suggest that prompts may not be com- pletely essential 39 40). SAMed 6 integrates LoRA [4 layers into the image encoder while eliminating the prompts. Building on this progress, H-SAM 42] employs hicrarchical decoding structure to optimize the process of fine-tuning with limited medical data and attains impressive results, In this work, we also adopt prompt-free strategy to decrease the adverse effects of inaccurate prompts, further optimizing the application of SAM in semi- sumi-super medical image segmentation.
2.3 Efficient SAM with Knowledge Distillation
Knowledge distillation (KD) is model compression technique that improves a lightweight stu- dent by transferring knowledge from a complex teacher [43]. When deploying KD to accelerate SAM [3]. the objective is to convey knowledge from the original, larger SAM to more compact and efficient SAM-like models. Considering the encoder-decoder architecture of SAM, KD strategies are typically divided into two approaches: distilling the entire SAM model or distilling only the image encoder. For example, MobíleSAM (0) distills the image embedding from the image encoder into a lightweight ViT encoder while replicating the prompt-guided decoder. Based on this method, EfficientSAM 155 achieves a great pereeddprfoomancee trade-off through masked image pretraining.
However, TinySAM [17] points out that the absence of mask- supervision for the student network
3

---

Image embedding
Frozen
Teacher SAM
Label supervision
L earnable
Knowledge distillation
Original encoder
Mask
decoder
X
P
Output distillation
Embedding distillation
DLW
Student SAM
x
PS
Efficient encoder
Mask decoder
Image embedding
Ground Truth
Batch data
P
Figure 2: Overview of the proposed knowledge distillation- based semi-supervised learning strategy for ESDE. For labeled data, the student SAM receives direct supervision from ground-truth masks via segmentation losses. For unlabeled data, hierarchical distillation transfers knowledge from the frozen teacher (encoder embeddings and decoder outputs) to the student via KL divergence. Moreover, DLW strategy automatically adjusts distillation intensity for better model optimization.
may lead to significant performance degradation and, therefore, proposes a full-stage distillation framework. Inspired by these methods, in this work, we leverage KD by transferring knowledge from the teacher model to refine the predictions on unlabeled samples during semi-supervised learning.
3 Preliminaries
Notations. Let X' HR denote the image space, where each medical image X' has resolution of H x W. The corresponding label space is defined as ) {l...., "x where each pixel is assigned one of C anatomical or pathological categories. In the supeiiupereised setting, we have two datasets:D and n contains M unlabeled samples, typically N < M. A SAM-based segmentation model fe X + can be decomposed as an encoder-decoder:
{af yf y D {> N MJ where DD contains N labeled samples
(1) where Ee, denotes the image encoder, Ge, denotes the mask decoder, and q is the query of the mask decoder. which is formed by concatenating the prompt embedding with the output tokens. Let θ = (0) li=1 be the set of all parameter tensors of model fe. The total parameter count is defined P(fe) Di=1 | where |e(*) denotes the number of scalar parameters in tensor ( In particular, if fe, and fe, are two models, P(fe,) P(fe,) means fe, has more parameters.
fx = Ge, (z),
Problem Definition. The goal of SSMIS is learn a segmentation function fe that approaches the performance of fully supervised model using limited annotations. This can be formulated as cemi-supervised expected risk minimization problem:
min ( Suuu (fo (xz), y²)] AE
(Cussu DE a. x;")]
where Coup is the supervised loss (e.g., Dice cross-entropy), Lumsp exploits unlabeled data via consistency learning, neeudo-labeling. or knowledge distillation, and balances their contributions.
We adopt a teacher-studen paradigm to tackle the above problem. The teacher model fe, provides supervision, while the student feg. which has fewer parameters (P(Je,) > P(Jeg)) learns from labeled data and the teacher's guidance on unlabeled samples. Hierarchical knowledge distillation and dynamic loss weighting allow the student to inherit both the teacher's visual priors and domain- specific knowledge, yielding high- perforperfomancc yet lightweight segmentation model.

---

4 Method
4.1
Overview
As shown in Figure 2 our SSMIS framework employs teacher model fer (original SAM) and student model feg (efficient SAM variant), both using the default prompt embedding for segmentation. The training process comprises two sequential stages: supervised fine-tuning and semi-supervised learning. In the first stage (supervised fine-tuming), both teacher and student are fine-tuned on the labeled dataset D For the teacher model fey. we apply the bow-rank-bsse (LoRA) fine-tuning strategy to the image encoder. preserving generic visual features while incorporating medical domain knowledge. The student model feg is fine-tuned simultaneously to learn task-specific representations with limited annotations. In the second stage (semi-supervised learning), the fine-tuned teacher frozen and guides the student on unlabeled data D Crucially, hierarchical distillation pipeline transfers different-level feature representations from teacher to student, while dynamic loss weighting adaptively balances the distillation intensity, enabling efficient and stable knowledge transfer. During testing, only the lightweight student model feg is used for inference, providing fast and memory- efficient predictions while maintaining high segmentation performance.
4.2 Fine-tuning Stage
As illustrated in Figure 3 our foundational teacher model is built upon the original SAM, and it is composed of three components: a ViT-based image encoder, a prompt encoder, and mask decoder. The image encoder is the most heavyweight part, comprising a significant proportion of total parameters. Specifically, we freeze all layers in the image encoder and add smaller, trainable bypass for each transformer block. These bypasses, which constitute the LoRA layers, first compress the transformer features into a low-rank space and subsequently reproject these condensed features to align with the channel dimensions of the output features in the frozen transformer blocks. During training, only the LoRA layers are updated, facilitating subtle yet impactful adjustments to the model and enabling efficient parameter adaptation with minimal memory overhead.
For the SSMIS task, the prompt encoder plays a crucial role in achieving more accurate seg- mentation results. Here, we update the default embedding, which refers to the embedding gen- erated by the prompt encoder when no prompts are provided, to enhance SAM's performance and eliminate the need for manual inputs. This strategy not only avoids the potential pitfalls of inaccurate prompts, but also enables automated medical diagnosis. For the mask decoder, we update all parameters. In summary, we adopt the same implementation as SAMed [6] to fine- tune the teacher model. During fine-tuning, utilize a supervised loss Coup as the fine-tuning stage optimization objective La. enabling the model to learn accurate segmentation boundaries from labeled medical data effectively. The fine- tuning optimization is formulated as:
Image encoder
#
Leomabbee
Tabel suparvision
decoder
spore
Imaae embooding
Deloxlt embodilng
LoRA layers
Figure 3: The pipeline for fine-tuning the teacher SAM. We freeze the image encoder and integrate LoRA layers into transformer blocks for efficient fine-tuning. Additionally, we update the prompt encoder using default embedding to enable prompt-free setting.
La(wx y() = Laup (p: v Caup (p/,, y Adiie Laice (p:, v.) + Aco Lco (c
(3)
where pf = fe(x)) denotes the model prediction for a labeled image, and yf is the corresponding ground-truth mask. Ldice and Lee represent the Dice loss and Cross-entropy loss, respectively, with Adice and Ace controlling their relative contributions.
To obtain an efficient SAM backbone, we replace the image encoder of the original SAM with a much smaller VIT, while the rest of the components remain unchanged. For this student SAM, similarly, we also retain the strategy of updating the default embedding. During fine-tuning, all parameters are optimized, utilizing the same loss function the teacher model.
5

---

4.3 Semi-supervised Learning Stage
After fine-tuning, we further optimize the model through sumi--ueereiee learnin. As shown in Figure our framework employs teacher-student architecture to strategically leverage both labeled and unlabeled data. Specifically, the teacher model remains frozen to preserve reliable knowledge and generalization capability, while the student model maintains full trainability to assimilate two learning signals: direct supervision from labeled data via ground- distillation from the teacher's embedding-lewe and logit-level for unlabeled data. In this work, the proposed distillation mechanism is implemented through the newly designed hierarchical distillation pipeline and dynamic loss weighting strategy, both of which are detailed subsequently.
annotations and knowledge
Hierarchical Knowledge Distillation. Inspired by MobileSAM [0],, we adopt the image encoder output as the distilled information. Since (x;) and (2; are feature maps of shape D x H' x W', we compute KL divergence for each spatial position after softmax along the channel dimension. Given a batch of N' unlabeled images {a:) C DD, the embedding-leve distillation loss is:
Lemb
N'. W Σ N'
Σ
H²
Σ
W'
DKL. (ET(M)): (
where (r( = mal (E:())) (a)),,)), Es(s)))... = (fft (xI,(pll4)
El= log 24, with v indexing spatial positions and d denoting the number of dimensions.
Although the global features extracted by the image encoder provide a broad understanding of the image structure, they insufficient for the precise requirements of pixel-level mask prediction. Segmentation tasks rely more on features near the output layer, which capture localized details and boundaries. Hence, the teacher's logit output is also selected distillation target provide fine-grained guidance. The logit-level distillation loss is defined as:
H
W
Liogit
DKL. (Gr(ar)) || (s(mr)).m.)
(5)
H W
v=1
where = (., (orrex/GGr (ET((E))))) (e::).g).... )r),,,,). = (Gs(E (.se))..s).... qs)....)
Combining the embedding and logit distillation losses helps the student better replicate the teacher's performance. Thus, the overall knowledge distillation loss function is given
Lad Xemb + Mtggi Clogit
(6)
where Aemb and Atogit are weight coefficients that balance the contributions of the embedding and logit distillation losses, respectively.
Dynamic Loss Weighting. To prevent the static knowledge of the frozen teacher from dominating the learning process, we design a dynamic loss weighting (DLW) strategy that automatically adjusts the distillation intensity based on comparative performance metrics. The key mechanism involves monitoring the supervised losses of both models on labeled data: the teacher's loss C.up serves as fixed reference benchmark after fine-tuning, while the student's loss LS reflects its current learning progress. At each epoch f, we update the distillation weight λka through conditional decay:
a (1-1).
S CT
Ad (t-1) kd
otherwise,
where a is a scaling factor (e.g., 0.95) for gradual decay and > 1.0 initiates strong guidance. Note that CT, only used to compute Aka and does not contribute to the backpropagation of gradients. DLW helps the student rely more the teacher's guidance in the early stages while gradually focusing on self-learning as training progresses, which aids in adapting unlabeled data and improving generalization to unseen data.
In summary, the overall optimization objective for semi-supervised learning is written
Lall Lsup Axd
(
Differing from S which denotes the supervised loss of the student model on labeled data summed over entire epoch, Coup here denotes the loss for each training iteration.

---

Table 1: Comparisons with SOTA semi-supervised segmentation methods on the LA dataset.
Scans used
Metrics
Scans used
Metries
Method
Labeled Unlabeled
DSCt
Jaccardf
95HDJ ASD
Labeled Unlabeled
DSC1
Jaccard 95HD, ASD
UA-MT 29
82.26
70,98
13.71
3.82
87.79
78.39
8.68
2.12
SASSNet
49
81.60
69.63
16.16
3.58
87.54
78.05
9.84
2.59
DTC 500
81.25
69.33
14.90
3.99
87.51
78.17
8.23
2.36
URPC 51
82.48
71.35
14.65
3.65
86.92
77.03
11.13
2.28
4(5%) 76(95%)
8(10%) 72(90%)
MC-Net
12
83.59
72.36
14.07
2.70
87.62
78.25
10.03
1.82
SS-Net 31
86.33
76.15
9.97
2.31
88.55
79.62
7.49
1.90
BCP 30
88.02
78.72
7.90
2.15
89.62
81.31
6.81
1.76
Ours
88.42
79.47
8.15
2.65
90.47
82.71
6.41
1.97
Table 2: Comparisons with SOTA semi-supervised segmentation methods on the Brats-2019 dataset.
Scans used
Metrics
Scans used
Metries
Method
Labeled Unlabeled
DSC1
Jaceard
95HD|
ASD
Labeled Unlabeled
DSC1
Jaceard
95HDJ ASDI
DAN 5
81.71
71.43
15.15
2.32
83,31
73.53
10.86
2.23
DTC
50
81.75
71.63
15.73
2.56
83.43
73.56
14.77
2.34
CPS 54
82.52
72.66
13.08
2.66
84.01
74.02
12.16
2.18
URPC
S
82.59
72.11
13.88
3.72
82,93
72.57
15.93
4.19
25(10%) 225(90%)
50(20%) 200(80%)
CPCL 55
83.36
73.23
11.74
1.99
83.48
74.08
9.53
2.08
AC-MT
56
83.77
73.96
11.37
1.93
84.63
74.39
9.50
2.11
MLRPL
5
84.29
74.74
9.57
2.55
85.47
76.32
7.76
2.00
Ours
85.14
75.56
8.58
2.57
86.46
77.21
7.64
2.21
Experiments
We validate our proposed method on three widely-used semi-supervised medical image segmentation datasets: the LA dataset [44), the Brats-2011 dataset (45), and the PROMISE12 dataset [46). Addi- tionally, to facilitate a comprehensive comparison with existing prompt-free medical SAM variants, we conduct experiments on the Synapse Kulti-Organ CT dataset [47]. For the foundational teacher SAM, we conduct all experiments based on the "ViT-B" version, while for the efficient student SAM, we replace the original image encoder with Tiny [88.. More details of the datasets, evaluation metrics, and method implementation, as well as additional experimental results such as the effectiveness of the fine-tuned teacher and model complexity analysis, are provided in the Appendix.
5.1 Comparison with Sate-of-the-Art Methods
Results on LA Dataset. We evaluate our method on the LA dataset against sfateoof-the-ar semi- supervised methods, including UA-MT 229]. SASSNet 44),, DTC (50), URPC MC-Net [52). SS-Net [3I], and BCP (30). using labeled ratios of 5% and 10%. As shown in Table our approach outperforms these competitors, achieving a 0.40% improvement in DSC with 5% labeled data and 0.85% with 10%. Additionally, other metries highlight its competitiveness, with gains of 1. 1.40% in Jaccard and 0.40 in 95HD under the 10% setting. It demonstrates the feasibility of using SAM as the backbone network with a default embedding and validates the effectiveness of the proposed knowledge distillation-base Figure 4 demonstrate that our method produces finer segmentation boundaries and better overall agreement with ground-truth annotations compared to existing approaches.
semi-supervised learning strategy. Moreover, qualitative results in
Results on Brats-2019 Dataset. Brain tumor segmentation is highly challenging due variations in tumor appearance and uncertain boundaries. To demonstrate the effectiveness of our approach, we conduct experiments on the Brats-2019 dataset with 10% and 20% labeled ratio, comparing it with
several methods. including DAN (33. DTC [50]. CPS 54 URPC [51]. CPCL (55) AC-MT [56). and MLRPL [57]. As evidenced Table our method achieves sfate-of-the-ae art performance on both labeled protocols. With 10% labeled data, we obtain 85. 14% DSC and 75.56% Jaccard scores, outperforning the strongest baseline (MIRPL) by 0.85% and 0.82%, respectively. This advantage further expands to 1.0% DSC and 0.89% Jaccard improvements under the 20% labeled setting. Furthermore, our approach reduces 95HD by 0.99 compared to MIRPL. with 10% labeled data, demonstrating a superior boundary adherence capability for complex tumor structures. The consistent
7

---

(a)
(b)
(c)
(d)
Figure 4: Visualization of segmentation results on the LA dataset with 10% labeled data. (a) Groud-
truth. (b) Ours results. (c) BCP results. (d) SS-Net results. (e) MC-Net results. (f) DTC results.
performance gains validate the effectiveness of our proposed hierarchical knowledge distillation and dynamic loss weighting strategy in handling ambiguous tumor margins.
Results on PROMISE12 Dataset. We also perform experiments on the PROMISE12 dataset with 20% labeled ratio against CCT (58], URPC [51], SS-Net (31), SLC-Net (59), SCP-Net 60), BCP 30, and ABD 28,, as well as ABD with 10% labeled ratio. Detailed results are shown the Appendix.
5.2 Ablation Studies
We conduct ablation studies to evaluate the impact of key components in our method, including combinations of knowledge distillation losses, the ratios of Xemb and Atogi (Eq ( fine-tuning the student model, and the proposed DLW strategy. All experiments here are performed on the LA dataset, with ablation results on other datasets provided in the Appendix.
Table Ablation study combinations of knowledge distillation losses.
Table 4: Ablation study on ratios of Xemb and Atogi with 10% labeled data.
Embedding Logit
Scans used
Metries
Metrics
Labeled Unlabeled DSC1
Juccurd] 95HD.
ASD.
Aemb
Auegit
DSCf
Jaceard?
9SHDJ
ASDJ
81.72
70.92
23.24
6.24
1/4
3/4
90.28
82.39
6.70
2.03
4(5%) 76(95%)
88.07
79.00
10.73
1/3
2/3
90.47
82.71
6.41
1.97
88.42
79.47
8.15
1/2
1/2
90.25
82.35
6.81
2.13
84.53
75.18
11.75
2/3
1/3
89.92
81.86
6.95
2.14
8(10%) 72(90%))
90.19
82.26
6.64
90.47
82.71
6.41
1/4
89.38
80.94
9.24
2.94
Table 5: Effectiveness of fine-tur fine-tuning in the student model.
Table 6: Effectiveness of the proposed DLW strategy.
Scans used
Metries
Method
Method
Labeled Unlabeled DSC
Jaccard?
95HDJ
ASD]
Labeled Unlabeled
DSCT
Jaccard]
951ID)
ASD]
w/o fine-tune
86.57
76.73
9.95
3.08
w/ DLW
88.00
78.94
8.10
2.67
4(5%) 76(95%)
4(5%) 76295%)
w/ fine-tune
88.42
79.47
8.15
2.65
w/ DLW
88.42
79.47
8.15
2.65
w/o fine tune
8(10%) 72(90%)
88
80.19
7.05
2.30
w/o DLW
8(10%) 72(90%)
90.02
81.97
7.04
2.25
w/ fine- tune
90.44
82.71
6.41
1.97
90.47
82.71
6.41
1.97
Different Knowledge Distillation Losses. Table examines the effectiveness of different combi- nations of knowledge distillation losses, including embedding loss and logit loss, to evaluate their contributions. The results reveal that the logit loss plays critical role, it closely aligns with the supervised information and directly impacts the evaluation metrics. Using the embedding loss alone shows weaker performance, while the logit loss alone yields noticeable improvements. However, combining both losses significantly enhances segmentation performance. These findings highlight the complementary roles of embedding loss, which enables the student encoder to learn semantic knowledge of image-dens, features and spatially structured relationships from the teacher encoder, and logit loss, which leverages the supervised alignment output to better align with the teacher.
Effeet of Balancing Embedding and Prediction Losses. explore the impact of different weight ratios between embedding loss and logit loss, conduct ablation study with 10% labeled data, as

---

Table 7: Comparisons with SAM-assisted semi-supervised segmentation methods on the LA dataset.
Scans used
Metrics
Scans used
Metrics Jaceardf 95HDL ASD1
Method
Labeled Unlabeled DSC1 Jaccardt 9SHD,
ASDI Labeled Unlabeled
DSC1
SemiSAM M
80.42
68.05
18.23
5.16
84,45
73.75
14,56
3.31
UP-SAM 62
84.06
72.90
13.78
3.14
4(5%) 76(95%)
8(10%) 72(90%)
SFR 12
87.95
78.83
9.23
2.89
89.99
81.93
7.07
2.04
Qurs
88.42
79.47
8.15
2.65
90.47
82.71
6,41
1.97
Table 8: Comparisons with SOTA prompt-free medical SAM variants on the Synapse dataset with 10% labeled data.
Metries
Method
Params
Aorta
Gallbladder
Kidney(l.) Kidney(R)
Liver Pancreas
Spleen
Stomach
DSC1
95HDJ
SAM Adapter S
131.5M
66,74
22.38
66.77
68.38
89.69
26.76
72.42
53.15
58.29
54.22
AM (D
112.5M
75.19
24.87
76.53
77.44
88.06
34.58
68.80
52.70
62.27
31.67
SAMed G
108. 8M
78.72
63.15
82.62
82.25
92.72
52.12
85.82
67.20
75.58
23.02
H-SAM
42
112.3M
79.65
59.76
82.71
82.14
91.73
47.48
86.35
75.31
75.68
21.34
Ours
10.1M
86.58
67.22
83,23
79.04
93.00
57.08
86.18
75.05
78.42
17.87
shown in T The results indicate that the balance between the two losses significantly affects segmentation performance. When the ratio Aemb Atogit is set to 1/3 2/3, the best performance is achieved. However, as Aemb increases, noticeable decline in performance is observed, with DSC dropping from 90.47% to 89.38% at 3/4: 1/4. This indicates that overemphasizing the embedding loss undermines the contribution of the logit loss, which is more aligned with the supervised signals.
Fine-tuning in Student. Motivated by БT), we apply fine-tuning to the student model prior to semi-supervised learning. This step helps the student better adapt to the target task by leveraging labeled data for initial optimization, creating more robust foundation for knowledge distillation. As shown in Table 5 fine-tuning reliably improves performance across all metrics, with significant gains observed in DSC and Jaccard scores. These results highlight the importance of initializing the student model with task-specific knowledge, which enhances its capacity to benefit from the subsequent semi-supervised learning process.
Dynamic Loss Weighting Strategy. In this work, we propose the DLW strategy to mitigate the potential over-reliance of the student model on the fixed outputs of the teacher model. To validate its effectiveness, we perform ablation experiments to evaluate its impact on segmentation performance. As shown in Table 6 incorporating DLW consistently enhances the results. For 5% labeled data, DLW improves the DSC to 88.42% and slightly reduces the ASD to 2.65. Similarly, for 10% labeled data, significantly decreases the ASD from 2.25 to 1.97. By dynamically adjusting the weight of the distillation loss during training, DLW allows the student model to initially leverage the teacher's guidance while gradually shifting focus toward self-learning. This adaptive mechanism supports effective utilization of unlabeled data and contributes to better generalization.
5.3 Comparisons with SAM-assisted Semi-supervised Methods
In this work, we directly SAM as the backbone network with default embedding and design a novel knowledge distillation- learning strategy. Therefore, we also conduct quantitative experiments on the LA dataset compare the proposed method with three existing SAM-assisted semi-supervised methods: SemiSAN AM [T]. UP-SAM (62), and SFR [12). It is important to highlight ConvNet backbone for semi-supervised
that these methods use SAM as an auxiliary component learning, and the outputs from the ConvNet are regarded as the final segmentation maps. As shown in T our method achieves the best performance, verifying that employing SAM as the segmentation pipeline to unleash its capability yields more substantial benefits than using it as an auxiliary module. This superiority can attributed to the fact that the Conv-Net-centric architectures of existing methods and their provision of unreliable prompts inevitably compromise the inherent segmentation potential of SAM, while our model with prompt-free setting successfully circumvents error accumulation in pseudo-label generation.

---

Spleen
Kidney (R)
Kidney(L)
Gallbladder
Liver
Stomach
Aorta
Pancreas
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(c)
Figure 5: Visualization of segmentation results on the Synapse dataset with 10% labeled data. (a) Groud-truth. (b) Ours results. (c) H-SAM results. (d) SAMed results.
5.4 Comparisons with Prompt-free Medical SAM Variants
For more comprehensive evaluation, further compare method with several prompt-free medical SAM variants, including SAM Adapter [3], AutoSAM [40], SAMed [6], and H-SAM [42]. all of which default the "VIT-B" version. Table 8 shows the quantitative results the multi-class task using the Synapse dataset with 10% labeled data. For fair comparison, we upsample 224 x 224 CT images to 512 x 512 input instead of directly using resolution of 512 512, to maintain consistency with the methods mentioned above. Notably, compared to these fully supervised methods, our method compresses the knowledge of teacher SAM (equivalent to SAMed) into a student SAM with more than 10 x fewer parameters, while also effectively leveraging a large amount of unlabeled data. can be observed that our method consistently outperforms the other methods across the majority of organs, achieving higher segmentation accuracy while significantly reducing inference costs. Moreover, Figure gives some qualitative results where our model yields smoother and more accurate segmentation regions compared to other methods.
Conclusion
this work, we explore a better adaptation SAM semi-supervised medical image segmentation. Capitalizing on advancements in lightweight SAM techniques, pioneer its deployment as the backbone network semi-supervised frameworks to fully leverage its segmentation capability. To address the incompatibility of conventional semi-s semi-supervised methods with the SAM backbone, develop a novel knowledge distillation based learning strategy that achieves hierarchical distillation by aligning the anatomical semantics of the encoder with the boundary details of the decoder, and incorporates dynamic loss weighting to progressively reduce the distillation intensity for better ex- ploitation of unlabeled data. In this way, the proposed method achieves higher segmentation accuracy while significantly reducing model parameters, enhancing the feasibility of clinical deployment. This technical paradigm for the practical implementation of foundational
work also establishes models in medical image segmentation.
Acknowledgements
This project is funded by the National Natural Science Foundation of China (62576006, 62076005, 62472004), Provincial Quality Project of Education in the New Era in 2023 (Postgraduate Education 2023lhpysfjd009), and the University Synergy Innovation Program of Anhui Province, China (GXXT- 2021-002, GXXT-2022-029). We sincerely thank Professor Bin Luo for his generous support of this project. We are grateful to Ke Zhu (LAMDA. Nanjing University) and Ming-Kun Xie (RIKEN AIP) for their valuable suggestions. We also thank the reviewers for their comments and suggestions. Finally, we acknowledge the University for providing computational resourees for this project.
Computing Platform of Anhui
10

---

References
[1] Geert Litjens. Thijs Kooi, Babak Ehteshami Bejuordi, Amaud Arindra Adiyoso Setio, Francesco Ciompi. Mohsen Ghafoorian, Jeroen Awm Van Der Laak, Bram Van Ginneken, and Clara Sánchez. survey on deep learning in medical image analysis. Medical Image Analysis, 42:60-88, 10600088,220 2017.
[2] Kevin Zhou, Hayit Greenspan, Christos Davatzikos, James S Duncan, Bram Van Ginneken, Anant Madabhushi, Jerry L Prince, Daniel Rueckert, and Ronald M Summers. A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises. Proceedings of the IEFE, 109(5):820-83) 838, 2021.
[3] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tote Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollar, and Ross Girshick. Segment anything. In ICCV, pages 4015-4026, 2023.
[4] Jun Ma, Yuting Hc, Feifei Li, Lin Han, Chenyu You, and Bo Wang. Segment anything in medical images. Nature Communications, 15(1))644, 2024.
[5] Junde Wu, Ziyue Wang. Mingxuan Hong. Wei Ji. Huazhu Fu, Yanuu Xu, Min Xu, and Yueming Jin. Medical sam adapter: Adanting segment anything model for medical image segmentation. Medical image analysis, 102:103547, 2025.
[6] Kaidong Zhang and Dong Liu. Customized segment anything model for medical image segmentation. arXiv preprint arXiv:2304. 13785, 2023.
Zhen Chen, Qing Xu, Xinyu Liu, and Yixuan Yuan. Un-sam: Universal prompt-free segmentation for generalized nuclei images. arXiv preprint arXiv:2402. 16663, 2024.
[8] Xian Lin, Yangyang Xiang, Li Yu, and Zenggiang Yan. Beyond adapting sam: Towards end-to-end ultrasound image segmentation via auto prompting. In MICCAI, pages 24 34, 2024.
[9] Qi Wu, Yuyao Zhang, and Marawan Elbatel. Self- ceffpprompting large vision models for few-shot medical image segmentation. In MICCAI Workshop, pages 156-167, 2023.
[10] Chumpeng Zhou, Kangjie Ning, Qianqian Shen. Sheng Zhou, Zhi Yu, and Haishuai Wang. Sam-sp: Self- prolpprompiin makes sam great again. arXiv preprint arXiv:2408. 12364, 2024.
[11] Yichi Zhang, Yuan Cheng, and Yuan Qi. Semisam: Exploring sam for enhancing semi- supervisd medical image segmentation with extremely limited annotations. arxiv preprint arXiv:2312.06316, 2023.
[12] Shumeng Li, Lei Qi. Qian Yu, Jing Huo, Yinghuan Shi, and Yang Gao. Stitching, fine-tuning, re- re-training: A sam-enabled framework for semi-superrised 3d medical image segmentation. IEEE Transactions on Medical Imaging, 2025.
[13] Kaiwen Huang. Tao Zhou, Huazhu Fu, Yizhe Zhang. Yi Zhou, Chen Gong, and Dong Liang. Learnable prompting sam-induced knowledge distillation for semi- superviee medical image segmentation. IEEE Transactions Medical Imaging, 2025.
[14] Nikhila Ravi, Valentin Gabeur, Yuan- Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.0 240.044400.00114 2024.
[15] Yunyang Xiong. Bala Varadarajan, Lemeng Wu, Xiaoyu Xiang, Fanyi Xiao, Chenchen Zhu, Xiaoliang Dai, Dilin Wang, Fei Sun, Forrest landola, al. Efficientsam: Leveraged masked image pretraining for efficient segment anything. In CVPR, pages 16111-16121 2024.
[16] Zhao, Wenchao Ding, Yongqi An. Yinglong Du Tao Yu, Min Li, Ming Tang, and Jinqiao Wang. Fast segment anything. arXiv preprint arXiv:2306. 12156, 2023.
[17] Han Shu, Wenshuo Li, Ychui Tang, Yiman Zhang, Yihao Chen, Houqiang Li, Yunhe Wang, and Xinghao Chen. Tinysam: Pushing the envelope for efficient segment anything model. In AAAI, volume 39, pages 20470-2000 20400, 2025.
[18] Ao Wang, Hui Chen, Zijia Lin, Jungong Han, and Guiguang Ding. Repvit: Revisiting mobile enn from vit perspective. In CVPR, pages 15909-15920, 2024.
[19] Haobo Yuan, Xiangtai Li. Lu Qi, Tao Zhang, Ming-Hsuan Yang. Shuicheng Yan, and Chen Change Loy. Mamba or rwky: Exploring high-quality and high-efficiency segment anything model. arXiv preprint arXiv:2406.19360 2024.
11

---

[20] Chaoning Zhang, Dongshen Han, Yu Qiao, Jung Uk Kim. Sung-Ho Bae, Seungkyu Lee, and Choong Seon Hong. Faster segment anything: Towards lightweight for mobile applications. arXiv preprint urXiv:2306. 14289, 2023.
[21] Chong Zhou, Xiangtai Li, Chen Change Loy, and Bo Dai. Edgesam: Prompt-in -he-ti-ttthooppp distillation for on-device deployment sam. arXiv preprint arXiv:2312.06660, 2023.
[22] Antti Tarvainen and Harri Valpola. Mean teachers better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. NeurlPS, 30, 2017.
[23] Zejia Weng. Xitong Yang. Ang Li, Zuxuan Wu, and Yu-Gang Jiang. Semi-supervised vision transformers. In ECCV, pages 605-620, 2022.
[24] Ming-Kun Xie, Jiahao Xiao, Hao-Zhe Liu, Gang Niu, Masashi Sugiyama, and Sheng-Jun Huang. Class- distribution-aware pseudo-labeling for semi-supervised multi-label learning. NeurlPS, 36:25731-25747, 2023.
[25] Jia-Hao Xiao, Ming-Kun Xie, Heng-Bo Fan, Gang Niu, Masashi Sugiyama, and Sheng-Jun Huang. Dual- decoupling learning and metric-adaptive thresholding for semi-supervised multi-label learning. In ECCV, pages 437-454, 2024.
[26] Hao-Zhe Liu. Ming-Kon Kun Xie, Chen-Chen Zong, and Sheng-Jun Huang. Asymmetric beta loss for evidence- based safe semi-supervised multi-label learning. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1909-1920, 2024.
[27] Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In ICML Workshop. volume 3. page 896, 2013.
[28] Hanyang Chi, Jian Pang, Bingfeng Zhang, and Weifeng Liu. Adaptive bidirectional displacement for semi-supervised medical image segmentation. In CVPR, pages 4070-4080, 2024.
[29] Lequan Yu, Shujun Wang, Xiaomeng Li, Chi-Wing Fu, and Pheng-Ann Heng. Uneertainty-awar self- ensembling model for semi-supervised 3d left atrium segmentation. In MICCAI, pages 605-613 2019.
[30] Yunhao Bai, Duowen Chen, Qingli Li, Wei Shen, and Yan Wang. Bidirectional copy-paste for semi- supervised medical image segmentation. In CVPR, pages 11514-11524, 2023.
[31] Yicheng Wu, Zhonghua Wu, Qianyi Wu, Zongyuan Ge, and Jianfei Cai. Exploring smoothness and class-separation for semi-supervised medical image segmentation. In MICCAI, pages 34-43, 2022.
[32] Mingjin Zhang, Qian Xu, Yuchun Wang, Xi Li, and Haojuan Yuan. Mirsam: multimodal lanionnl segment anything model for infrared small target detection. Visual Intelligence, 3(1):1-13, 2025.
[33] Dongjie Cheng. Ziyuan Qin, Zekun Jiang, Shaoting Zhang, Qicheng Lao, and Kang Li. Sam on medical images: A comprehensive study on three prompt modes. arXiv preprint arXiv:2305.0003 50035, 2023.
[34] Chuanfei Hu, Tianyi Xia, Shenghong Ju, and Xinde Li. When meets medical images: An inves- tigation of segment anything model (sam) on multi-phase liver tumor segmentation. arXiv preprint arXiv:2304.08500 085000 2023.
[35] Tassilo Wald, Saikat Roy, Gregor Kochler, Nico Disch, Maximilian Rouven Rokuss, Julius Holzschuh, David Zimmerer, and Klaus Maier-Hein. Sam. md: Zero-shot medical image segmentation capabilities of the segment anything model. In Medical Imaging with Deep Learning, short paper track, 2023.
[36] Sovesh Mohapatra, Advait Gosai, and Gottfricd Schlaug. Sam vs bet: A comparative study for brain extrac- tion and segmentation of magnetic resonance images using deep learning. arXiv preprint 240.0433, 2023.
empirical study on generalization, robustness and adaptation. In MICCAI, pages 234-244, 2023. [37] An Wang, Mobarakol Islam, Mengya Xu, Yang Zhang, and Hongliang Ren. Sam meets robotic surgery: an
[38] Bin Xie, Hao Tang, Bin Duan, Dawen Cai, and Yan Yan. Masksam: Towards auto-prompp sam with mask classification for medical image segmentation. arXiv preprint arxiv:2403. 14103, 2024.
[39] Weijia Feng, Lingting Zhu, and Lequan Yu. Cheap lunch for medical image segmentation by fine-tuning sam on few exemplars. In MICCA/ Workshop, pages 13-22, 2023.
[40] Xinrong Hu, Xiaowei Xu, and Yiyu Shi. How to efficiently adapt large segmentation model (sam) to medical images. arXiv preprint 1377:1306.13331, 2023.
12

---

[41] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, Weizhu Chen, al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
[42] Zhiheng Cheng, Qingyue Wei, Hongru Zhu, Yan Wang, Liangqiong Qu, Wei Shao, and Yuyin Zhou. Unleashing the potential of sam for medical adaptation via hicrarchical decoding. In CVPR, pages 3511-3522, 2024.
[43] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. NeurlPS Workshop, 2015.
[44] Chen Chen, Wenjia Bai, and Daniel Rueckert. Multi-task learning for left atrial segmentation on ge-mri. In Statistical Atlases and Computational Models of the Heart, Atrial Segmentation and LV Quantification Challenges: 9th International Workshop. STACOM 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers 9, pages 292-301, 2019.
[45] Spyridon (Spyros) Bakas. Brats miccai brain tumor dataset, 2020.
[46] Geert Litjens, Robert Toth, Wendy Van De Ven, Caroline Hoeks, Sjoerd Kerkstra, Bram Van Ginneken, Graham Vincent, Gwenael Guillard. Neil Birbeck, Jindang Zhang, et al. Evaluation of prostate segmentation algorithms for mri: the promise 12 challenge. Medical Image Analysis, 18(2):359- -333333, 2014.
[47] Bennett Landman, Zhoubing Xu, Igelsias, Martin Styner, Thomas Langerak, and Arno Klein. Miccai multi-atlas labeling beyond the cranial vault-workshop and challenge. In MICCA/ Workshop, volume 5, page 12, 2015.
[48] Kan Wu, Jinnian Zhang, Houwen Peng, Mengchen Liu, Bin Xiao, Jianlong Fu, and Lu Yuan. Tinyvit: Fast pretraining distillation for small vision transformers. In ECCV, pages 68-85, 2022.
[49] Shuailin Li, Chuyu Zhang, and Xuming Hc. Shape-aware semi-supervised 3d semantic segmentation for medical images. In MICCAI, pages 352-561, 2020.
[50] Xiangde Luo, Jieneng Chen, Tao Song, and Guotai Wang. Semi-supervised medical image segmentation through dual-task consistency. In AAAI, volume 35, pages 8801-8809, 2021.
[51] Xiangde Luo, Wenjun Liao, Jieneng Chen, Tao Song, Yinan Chen, Shichuan Zhang, Nianyong Chen, Guotai Wang, and Shaoting Zhang. Efficient semi-supervised gross target volume of nasopharyngeal carcinoma segmentation via uncertainty rectified pyramid consistency. In MICCAI. pages 318-329, 2021.
[52] Yicheng Wu, Minfeng Xu, Zongyuan Ge, Jianfei Cai, and Lei Zhang. Semi-supervised left atrium segmentation with mutual consistency training. In MICCAI, pages 297-306, 2021.
[53] Yizhe Zhang, Lin Yang, Jianxu Chen, Maridel Fredericksen, David P Hughes, and Danny Z Chen. Deep adversarial networks for biomedical image segmentation utilizing unannotated images. In MICCAI, pages 408 416, 2017.
[54] Xiaokang Chen, Yuhui Yuan, Gang Zeng, and Jingdong Wang. Semi-superrised semantic segmentation with cross pscudo supervision. In CVPR, pages 2613-2622, 2021.
[55] Zhe Xu, Yixin Wang, Donghuan Lu, Lequan Yu, Jiangpeng Yan, Jie Luo, Kai Ma. Yefeng Zheng, and Raymond Kai-yu Tong. All-around real label supervision: Cyclic prototype consistency learning for semi semi-super medical image segmentation. IEEE Journal of Biomedical and Health Informatics. (677):3144-1184 2022.
[56] Zhe Xu, Yixin Wang, Donghuan Lu, Xiangde Luo, Jiangpeng Yan, Yefeng Zheng, and Raymond Kai-yu Tong. Ambiguity selective consistency regularization for mean-teacher supervisd medical image segmentation. Medical Image Analysis, 18:102880, 2023.
[57] Jiawei Su, Zhiming Luo, Sheng Lian, Dazhen Lin, and Shaozi Li. Mutual learning with reliable pseudo label for semi-supervised medical image segmentation. Medical Image Analysis, 94: 103111, 2024.
[58] Yassine Ouali, Céline Hudelot, and Myriam Tami. Semi-supervised semantic segmentation with cross- consistency training. In CVPR, pages 12674-12684 2020.
[59] Jinhua Liu, Christian Desrosiers, and Yuanfeng Zhou. Semi-supervised medical image segmentation using cross-model pscudo-supervision with shape awareness and local context constraints. In MICCAI, pages 140-150, 2022.
[60] Zhenxi Zhang, Ran Ran, Chunna Tian, Heng Zhou, Xin Li, Fan Yang, and Zhicheng Jiao. Self-aware and cross sample prototypical learning for semi- superviue medical image segmentation. In MICCAL pages 192-201, 2023.
13

---

[61] Zhaowei Cai, Avinash Ravichandran, Paolo Favaro, Manchen Wang. Davide Modolo, Rahul Bhotika, Zhuowen Tu, and Stefano Soatto. Semi-supervised vision transformers scale. NewrIPS, NeurlPS, 35:25697-25710, 2022.
[62] Wenjing Lu, Yi Hong. and Yang Yang. Up-sam: Uncertainty-nnformee adaptation of segment anything model superviie medical image segmentation. BIBM, pages 2256-2261, 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The main claims made the abstract and introduction accurately reflect the paper's contributions and scope, clearly outlining both the methodological innovations and empirical findings. Subsequent sections provide firm theoretical grounding and robust experimental validation for each claim.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of this work in the supplementary material.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
15

---

Answer:
[NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification:
The information needed to reproduce the main experimental results is intro-
duced in Section 4 Section and the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
16

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
We are currently not providing open access to the data and code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ praips.cc/pulli/guuudee CoodeSumeeieeeeee ssi iessonPolll mollicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All the training and test details can be seen in Section 5 and the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Our experiment did not include error bars, confidence intervals, or other statistical significance tests.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
17

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Sufficient information on the computer resources can be seen in the supple- mentary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the wttpuuuilcccpppppie/eeeccc
NeurIPS Code of Ethics
Answer: [Yes]
Justification:
The research conducted in the paper fully compliant with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses both the potential positive societal impacts and negative societal impacts of the work in the supplementary material.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
18

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite the original paper that produced the code package or dataset.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
19

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
20

---

16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy buctps://neure wwwparippps..coooeeeeeeeee for what should or should be described.
21


Reviews:
[{"review_id": "FTEuBYXCOA", "reviewer": "Reviewer_Href", "summary": "The paper presents a semi-supervised segmentation method for the Segment Anything Model (SAM) which distills a standard SAM segmentation backbone to a more efficient one, using both labeled and unlabeled images. As in standard Teacher-to-Student distillation approaches for semi-supervised learning (SSL), the predictions of the Teacher for unlabeled images are used as pseudo-supervision to train the student, in addition to a standard supervised segmentation loss on labeled images. Two main modifications are made to this approach. First, a distillation loss is applied to both the output of the image encoder and the segmentation decoder. Second, the distillation transfers knowledge from the Teacher to a more efficient Student backbone. The proposed method also introduces a fine-tuning step to learn task-specific features and a dynamic adjustment of the distillation loss during SSL training. The method is evaluated on Left Atrium (LA), Brats-2019, PROMISE12 and Synapse datasets, achieving a better performance than recent SSL approaches for medical image segmentation.", "questions": "* The proposed method starts with a pre-trained SAM model which is fine-tuned to specific medical image segmentation tasks. Why not start with a model pre-trained on medical images, such as MedSAM ?\n\n* Can authors provide more information on the default embeddings for their mask decoder (size, which decoder inputs, etc.) ?  \n\n* In eq (3), how do you compute the KL divergence between feature maps? Do you add a learnable projector + softmax to map features into probability distributions?\n\n* For dynamic loss weighting, two factors can impact the performance difference between the Teacher and Student: 1) the model capacity (the Teacher has a greater capacity), and 2) access to training gradient from unlabeled images (only the Student). How can you separate these two factors when choosing lambda_kd?\n\n* In Section 4.1: \" As shown in Table 1, our approach outperforms these competitors\". Technically, BCP is better in terms of surface-to-surface error (HD and ASD).\n\n* Instead of the proposed method, it would also be possible to fine-tune the SAM-based Teacher with both labeled and unlabeled data, and only after perform distillation. Has this approach been tested? If so, how does it compare?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n* Existing works typically use Teacher-to-Student knowledge distillation for learning a more efficient model or to obtain pseudo-labels for unlabeled examples. As mentioned in the paper, a contribution of the proposed method is to do both these steps in a single framework.\n\n* The proposed method achieved outstanding performance on a varied set of segmentation datasets.\n\n* The paper is well-written and easy to follow.\n\nWeaknesses:\n\n* Despite the contributions mentioned above, the methodological novelty of the proposed approach remains somewhat limited, as it combines two well-known distillation strategies in a single framework. \n\n* Although the method outperforms strong SSL baselines, the comparison is a bit tricky since most (if not all) of these baselines are based on a CNN backbone, while the method employs a transformer (ViT-B) backbone. \n\n* Some recently published papers report a better performance than those in the paper, for example: \"DyCON: Dynamic Uncertainty-aware Consistency and Contrastive Learning for Semi-supervised Medical Image Segmentation\". I understand that the paper was put on Arxiv one month before the NeurIPS deadline, so I do not blame authors for this omission.\n\n* The ablation study shows that the method's performance is affected by the choice of hyperparameters, such as lamba_out and lambda_emb. However, unless I missed that part, it is unclear how these hyperparameters were selected (if a validation set was employed and if they were tuned specifically for each dataset).", "was_revised": true, "final_justification": "The authors have addressed my main concerns in their rebuttal.", "rebuttal": "# To Reviewer Href\n\nThank you for your thoughtful and constructive comment on our paper.\n\n## Weakness1: Limited novelty of combining two well-known distillation strategies.\n\nAnswer: Our main contribution lies in the innovation of a semi-supervised medical image segmentation framework, which is designed to be simple and effective. Although our approach indeed combines two existing distillation strategies, feature-level and output-level distillation, we believe that this multi-level distillation strategy offers clear benefits for optimizing the student model. Specifically, output-level distillation offers consistent supervision for the final predictions, while feature-level distillation complements it by aligning intermediate features, facilitating more detailed and stable knowledge transfer. Experimental results across multiple medical datasets validate that this simple design is highly effective in improving model performance. \n\n\n## Weakness2: Challenges of comparison with CNN-based SSL baselines.\n\nAnswer: Currently, most baseline methods for semi-supervised medical image segmentation are based on CNN backbones. These methods typically achieve high performance and are widely recognized in this field. According to our investigation, although some studies attempt to apply ViT to semi-supervised medical image segmentation (e.g., 3D-ViT), more attention in this area has been given to strategies that combine CNN and ViT, such as SemiCTrans and SemiTC. Considering time constraints and aiming to clearly demonstrate the effectiveness of our method, we choose to compare it with the aforementioned approaches. In addition, we extend a representative ViT-based semi-supervised learning method (Semi-ViT) to the medical image segmentation task and compare it with our method under a unified experimental setting (see Table 1).\n\n### Table 1: Comparison with ViT-based methods on the BraTS2019 dataset with 10% labeled data.\n\n| Method       | DSC ↑   | Jaccard ↑ | 95HD ↓ | ASD ↓ |\n|--------------|---------|-----------|--------|--------|\n| 3D-ViT       | 82.56   | 72.62     | 13.22  | 2.33   |\n| SemiTC       | 81.60   | -         | 13.35  | 3.04   |\n| SemiCTrans   | 83.62   | 73.06     | 9.42   | 2.63   |\n| Semi-ViT     | 81.03   | 70.51     | 15.25  | 4.20   |\n| Ours         | 85.14   | 75.56     | 8.58   | 2.57   |\n\nFurthermore, to our knowledge, we are the first to adopt an efficient SAM based on TinyViT as the backbone for a semi-supervised framework. Therefore, we additionally provide comparison experiments between our method and several medical SAM variants in Table 8 of our paper, which indirectly validates the performance advantage of our model over other ViT-based architectures.\n\n\n## Weakness3: Lack of comparison with recently published papers.\n\nAnswer: Thanks for your reminder and suggestion. We will carefully follow recently published papers such as DyCON in our future work.\n\n\n## Weakness4: Hyperparameter selection process for $\\lambda_{out}$ and $\\lambda_{emb}$.\n\nAnswer: We did perform hyperparameter tuning for $\\lambda_{out}$ and $\\lambda_{emb}$ using validation sets. Ablation study results on the LA dataset and the PROMISE12 dataset (Section 3 of the supplementary material) indicate that the model achieves optimal performance when $\\lambda_{emb}$ : $\\lambda_{out}$ is set to 1/3 : 2/3. We did not deliberately tune these hyperparameters for each individual dataset. Instead, we employed a unified strategy to ensure generalization and robustness of the method.\n\n\n## Question1: Model selection.\n\nAnswer: The primary focus of this work is to propose a novel semi-supervised medical image segmentation framework along with an effective learning strategy. We believe that our method is not only applicable to SAM, but also has the potential to be transferred to other foundational vision models. Therefore, we choose to start from the original SAM to better reflect the universality of the model and to evaluate the effectiveness of the proposed framework in semi-supervised medical image segmentation. Under this innovative framework design, the aforementioned foundational model is of course replaceable. On the one hand, MedSAM is a specialized medical segmentation model obtained by fine-tuning the original SAM on a large amount of medical image data. In theory, using it as the initial teacher model could potentially further enhance the performance of our method. On the other hand, to our knowledge, MedSAM relies on bounding boxes as prompt inputs, while our method addresses the issue of \"inaccurate prompts degrading SAM performance\" by feeding no prompts and updating the default embedding during training. This is another reason why we chose to build upon the original SAM rather than directly adopting MedSAM.\n\n\n## Question2: Details on default embeddings.\n\nAnswer: When no prompts are provided, the prompt encoder outputs a dense prompt embedding with the shape (batch_size, embed_dim, H, W), where embed_dim matches the channel dimension of intermediate features in the mask decoder and the image embeddings from the Image Encoder, and H, W denote the spatial dimensions. This prompt dense embedding is obtained by broadcasting a learnable default embedding, which is then added to the image embeddings and fed into the mask decoder to assist in generating the final mask predictions. Thanks for your reminder and suggestion. We will include a detailed description of the default embedding in the future version of our paper.\n\n\n## Question3: KL divergence on feature maps.\n\nAnswer: In Equation (3), we apply a softmax operation along the channel dimension of the feature maps to convert the feature representation at each spatial location into a probability distribution. Specifically, let $\\mathbf{s}$ and $\\mathbf{t}$ denote the intermediate feature maps from the student and teacher models, respectively. We first compute their corresponding probability distributions as follows:\n\n$$\np_s = \\mathrm{Softmax}(\\mathbf{s}), \\quad\np_t = \\mathrm{Softmax}(\\mathbf{t})\n$$\n\nThen, the KL divergence between the student and teacher distributions is computed at each spatial location and averaged across the entire feature map and batch:\n\n$$ L_{\\mathrm{embedding}} = D_{KL}({p_s} \\mid\\mid p_t) = \\sum p_s \\log\\left(\\frac{p_s}{p_t}\\right)   $$\n\nNo additional projection layers are introduced in this process, in order to avoid introducing extra hyperparameters.\n\n\n## Question4: Design of dynamic loss weighting.\n\nAnswer: Our designed dynamic loss weighting strategy automatically adjusts the distillation weight based on the comparison of the supervised loss performance between the student and teacher models on labeled data. Specifically, the teacher model, which has a larger capacity, is fine-tuned and then frozen at the start of training to serve as a relatively stable reference. The student model, having a smaller capacity, continuously updates its parameters during training using both labeled and unlabeled data, progressively improving its performance. At the early stage of training, since the student model's performance is weaker, its loss is typically higher than that of the teacher model, and at this time, the distillation weight $\\lambda_{kd}$ remains unchanged. As training progresses, when the student's performance approaches or surpasses that of the teacher, that is, when the student's loss is lower than the teacher's loss, the distillation weight $\\lambda_{kd}$ is multiplied by a scaling factor and gradually reduced.\n\n\n## Question5: Comparative experimental results on the LA dataset.\n\nAnswer: Out of eight evaluation metrics under two ratio settings, we achieve the best performance on five metrics. Therefore, the statement \"As shown in Table 1, our approach outperforms these competitors\" in the paper is intended to reflect the overall and comprehensive performance. In fact, our main contribution lies in the innovation of the framework design and learning strategy, whereas BCP focuses more on detailed data processing techniques.\n\n\n## Question6: Fine-tuning the teacher model with both labeled and unlabeled data.\n\nAnswer: Thanks for your reminder and suggestion. We carefully consider fine-tuning the teacher SAM using both labeled and unlabeled data and design the following new training pipeline. Specifically, we first perform preliminary fine-tuning on the teacher model using the labeled data. Then, the fine-tuned teacher is used to generate high-confidence pseudo-labels for the unlabeled data. Finally, we iteratively optimize the teacher model using a mixture of labeled and unlabeled data, supervised respectively by ground-truth and the previously generated pseudo-labels. However, the experimental results (see Table 2) indicate that this approach yields limited performance gains for the teacher SAM. We attribute this to the instability in the quality of pseudo-labels, which may introduce noisy information and mislead the optimization process. In addition, this method incurs substantial computational overhead and involves a more complex training pipeline, making it less efficient overall.\n\n### Table 2: Comparison between fine-tuning the teacher with all data and fine-tuning the teacher with only labeled data using DSC as the evaluation metric.\n\n| Dataset   | Label Ratio | All Data | Labeled Data |\n|-----------|-------------|----------|--------------|\n| LA        | 5%          | 85.61    | 86.22        |\n|           | 10%         | 88.37    | 88.54        |\n| PROMISE12 | 10%         | 84.83    | 85.39        |\n|           | 20%         | 86.59    | 86.00        |"}, {"review_id": "cpd8wDcbg7", "reviewer": "Reviewer_tKUd", "summary": "This paper proposes a SAM-based Semi-supervised learning method to address that the direct application of SAM-based methods to medical images often results in suboptimal performance. The authors propose a teacher-student framework that utilises an original SAM, which is rich in prior knowledge, as the teacher to optimise the student SAM backbone through hierarchical knowledge distillation and a dynamic loss weighting strategy.", "questions": "1.\tThe Brats dataset is multiclass (edema, NET, ET), so what are the tumour regions focused on in the experiment? \n\n2.\tAlso, does the mask decoder generate multiple masks? Or, as in the VFM setting, is it trained with an individual class at a time with a binary mask?\n\n3.\tIsn’t mitigating the prompt encoder during the mask generation like removing a fundamental block from SAM? What will happen if a prompt encoder is used? Is there a significant performance gain to be had by not using a prompt encoder?", "limitations": "Yes, mentioned in the supplementary material.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1.\tThe paper is easy to read, and the motivation is clear.\n\n2.\tUtilising efficient SAM is an interesting application for semi-supervised medical image segmentation.\n\n3.\tThe authors have conducted extensive experiments, along with ablation studies.\n\n4.\tResults show the method achieves SOTA performance.\n\nWeaknesses:\n\n1.\tIn the experimental evaluation section, the most recent SOTA in SSMIS is missing, for example, MC-Net+ [1], Co-BioNet [2], Correlation-based switching mean teacher for semi-supervised medical image segmentation [3].\n\n2.\tAdditionally, comparisons with SAM-variants for medical image segmentation are lacking, for example, MedSAM and SAM-Med2D.\n\n3.\tAuthors mentioned, “The success of efficient SAM variants inspires us to consider whether they can serve as backbone networks in semi-supervised learning frameworks, replacing traditional CNN-based backbones to fully leverage their capabilities”, yet the complexity analysis between SSMIS and the proposed method is missing.\n\n4.\tDetails on experimental result reproducibility and code repository are missing, despite the criteria for this being marked as ‘Yes’ in the checklist.\n\n[1] Y. Wu, Z. Ge, D. Zhang, M. Xu, L. Zhang, Y. Xia, J. Cai, Mutual consistency learning for semi-supervised medical image segmentation, Med. Image Anal. 81 (2022) 102530. \n\n[2] H. Peiris, M. Hayat, Z. Chen, G. Egan, M. Harandi, Uncertainty-guided dual-views for semi-supervised volumetric medical image segmentation, Nat. Mach. Intell. (2023) 1–15.\n\n[3] G. Deng, H. Sun, W. Xie, Correlation-based switching mean teacher for semi-supervised medical image segmentation, Neurocomputing (2025) 129818.", "was_revised": true, "final_justification": "The authors have adequately addressed the majority of my concerns. \nThe work is technically sound and opens up new directions for prompt-guided learning in limited-data medical imaging scenarios.\nHowever, the lack of clarity on some practical aspects and reproducibility remains; I hope authors will release their codes upon paper acceptance.", "rebuttal": "# To Reviewer tKUd\n\nThank you for your thoughtful and constructive comment on our paper.\n\n## Weakness1: Missing recent SOTA methods in experimental evaluation.\n\nAnswer: Thanks for your reminder and suggestion. Here, we compare our method with several approaches you provided (see Table 1). It is worth noting that CS-MT adopts a relatively complex design, incorporating multiple commonly used techniques such as diverse data augmentation strategies, dual-teacher models, and consistency regularization. However, our proposed method is compatible with these strategies. If integrated with existing semi-supervised segmentation approaches (e.g., BCP and ABD), it has the potential to further improve performance.\n\n### Table 1: Comparisons with some semi-supervised segmentation methods on the LA dataset.\n\n| Method       | Labeled  | Unlabeled  | DSC ↑  | Jaccard ↑ | 95HD ↓ | ASD ↓ |\n|--------------|----------|------------|--------|-----------|--------|--------|\n| MC-Net+      | 4 (5%)   | 76 (95%)   | -      | -         | -      | -      |\n| Co-BioNet    |          |            | 83.59  | -         | 21.18  | -      |\n| CS-MT        |          |            | 89.39  | 80.89     | 6.13   | 2.02   |\n| Ours         |          |            | 88.42  | 79.47     | 8.15   | 2.65   |\n| MC-Net+      | 8 (10%)  | 72 (90%)   | 88.96  | 80.25     | 7.93   | 1.86   |\n| Co-BioNet    |          |            | 89.20  | -         | 6.44   | -      |\n| CS-MT        |          |            | 91.00  | 83.55     | 5.51   | 1.62   |\n| Ours         |          |            | 90.47  | 82.71     | 6.41   | 1.97   |\n\nMoreover, the selection criteria for the comparison methods in our paper are primarily based on either achieving the best performance on the corresponding datasets (e.g., BCP [CVPR, cited 322 times]) or being representative approaches (e.g., URPC [MICCAI, cited 323 times]), covering the main paradigms of current semi-supervised medical image segmentation. Therefore, comparison with these methods also demonstrates the effectiveness of our proposed approach. We will further include the methods listed in Table 1 in the future version of our paper.\n\n\n## Weakness2: Lack of comparison with medical SAM variants.\n\nAnswer: In Table 8 of our paper, we compare our method with several state-of-art medical SAM variants, such as SAMed and H-SAM. We highly acknowledge the contributions of other medical SAM variants like MedSAM and SAM-Med2D to medical image segmentation. These methods significantly enhance the adaptability of SAM to medical tasks through extensive annotated data. However, it is worth noting that the focus of this work lies in exploring the more effective application of SAM in semi-supervised medical image segmentation, rather than merely improving its performance on medical tasks. Therefore, we adopt the original SAM model without complex medical-specific tuning as the initial teacher model, emphasizing innovation at the framework level.\nHere, we also provide a comparison with MedSAM (see Table 2), where MedSAM uses ground-truth-derived bounding boxes as prompts.\n\n### Table 2: Comparisons with MedSAM on the Synapse dataset.\n\n| Method   | Aorta | Gallbladder | Kidney(L) | Kidney(R) | Liver | Pancreas | Spleen | Stomach | DSC↑  | 95HD↓  |\n|----------|-------|-------------|-----------|-----------|-------|----------|--------|---------|-------|--------|\n| MedSAM   | 70.16 | 22.44       | 79.08     | 64.63     | 76.38 | 52.31    | 73.07  | 79.10   | 64.65 | -      |\n| Ours     | 86.58 | 67.22       | 83.23     | 79.04     | 93.00 | 57.08    | 86.18  | 75.05   | 78.42 | 17.87  |\n\n\n## Weakness3: Complexity analysis between SSMIS baselines and Our method.  \n\nAnswer: We acknowledge that the paper does not explicitly provide a comparison of model complexity between our proposed method and the SSMIS baselines. It is important to note that SSMIS predominantly employs CNN-based backbone networks, whereas our study is the first to incorporate an efficient SAM as the backbone within the SSMIS framework. Consequently, there exist inherent architectural differences between our approach and the SSMIS baselines. To ensure a fair comparison, we selected representative and high-performing SSL baselines from the current literature for experimental evaluation. Furthermore, to further validate the effectiveness of our method, we supplemented the paper with comparative experiments against several medical SAM variants in Table 8. \nHere, we compare the efficient SAM backbone used in our method with the V-Net backbone commonly adopted in SSMIS baselines. V-Net relies on a 3D convolutional architecture, whose computational complexity can be expressed as:\n$$\n\\mathcal{O}(C^2 \\cdot K^3 \\cdot D \\cdot H \\cdot W)\n$$\nwhere $C$ denotes the number of channels, $K$ is the kernel size, and $D$, $H$, and $W$ refer to the depth, height, and width of the input volume, respectively. In contrast, our efficient SAM backbone is based on 2D image input and leverages a self-attention mechanism. Its computational complexity is given by:\n$$\n\\mathcal{O}(L \\cdot N^2 \\cdot d)\n$$\nwhere $L$ is the number of transformer layers, $N$ is the number of image patches, and $d$ is the dimension of each token. Moreover, our efficient SAM backbone contains 9.51M parameters, while V-Net has 9.44M parameters.\n\n\n## Weakness4: Reproducibility details and code.\n\nAnswer: In the checklist, we marked \"Yes\" only for \"Experimental result reproducibility,\" as we have provided extensive implementation details in both the paper and the supplementary material. These include dataset sources, preprocessing procedures, model architecture, training strategies, hyperparameter settings, and evaluation metrics. We are confident that other researchers can reproduce our results based on these settings. Moreover, our method is a simple yet effective framework rather than a complex combination of modules, so it does not require excessive hyperparameter tuning and is easy to implement.\nAs for the code repository, we marked it as \"No\" because the code was not fully organized at the time of submission. We have now completed most of the code organization and plan to release it publicly upon paper acceptance. The code repository link will also be included in the final version to further enhance the reproducibility of our method.\n\n\n## Question1: Processing of the BraTS dataset.\n\nAnswer: Our experimental setup on the BraTS 2019 dataset follows the existing semi-supervised medical image segmentation work [1]. We use only the FLAIR modality and merge the edema, NET, and ET regions into a single foreground class to perform a binary segmentation task of the whole tumor. This setup is widely recognized and adopted in semi-supervised segmentation tasks and helps standardize comparison across different studies.\n\n\n## Question2: Output of mask decoder.\n\nAnswer: The mask decoder does not generate multiple masks simultaneously. While the default SAM mask decoder uses multiple mask tokens to predict multiple candidate segmentation results concurrently, in medical scenarios, we follow the modification of the SAM segmentation head proposed in SAMed [2]. Specifically, the mask decoder output is adjusted to produce segmentation maps for k semantic classes (including background), and the final semantic label map is obtained via softmax followed by argmax. Therefore, unlike the VFM setting where the model is trained with one class at a time using binary masks, our model performs multi-class segmentation in a single forward pass.\n\n\n## Question3: Processing of prompt encoder.\n\nAnswer: We use the prompt encoder during mask generation and certainly do not remove it. Specifically, to mitigate the negative effects of inaccurate prompts on SAM's performance, we choose not to provide any prompts, allowing the prompt encoder to generate a default embedding, which is then fed into the mask decoder together with image embedding. During training, we further update this default embedding so that it gradually adapts to the feature distribution of medical images, thereby improving segmentation performance. It is worth noting that the prompt encoder is the key components of SAM, and completely removing it would break the original model architecture and severely degrade overall performance. In contrast, our approach preserves the full structure of SAM while effectively avoiding the negative impact caused by inaccurate prompts.\n\n\n[1] Luo X, Wang G, Liao W, et al. Semi-supervised medical image segmentation via uncertainty rectified pyramid consistency[J]. Medical Image Analysis, 2022, 80: 102517.\n\n[2] Zhang K, Liu D. Customized segment anything model for medical image segmentation[J]. arXiv preprint arXiv:2304.13785, 2023."}, {"review_id": "KAhNF7dmMt", "reviewer": "Reviewer_yrJD", "summary": "Existing Challenge 1: Annotated medical imaging data is scarce and expensive to obtain.  Approach: Leverage unlabeled data for semi-supervised learning to reduce reliance on annotated data.\n\nExisting Challenge 2: Although foundational vision models like SAM demonstrate strong generalization in general segmentation tasks, their direct application to medical images often results in suboptimal performance.  Approach: Fine-tune the original SAM using a small amount of labeled data to serve as a teacher network, and then use knowledge distillation to train a more lightweight SAM variant (student network) with the help of unlabeled data.", "questions": "1.\tEffectiveness of the teacher network:\nKnowledge distillation is essentially a model compression technique where the performance of the student network is inherently dependent on the quality of the teacher. If the teacher network performs poorly, the student network is unlikely to perform well either. Therefore, the paper should provide evidence of the teacher network’s effectiveness. \n\n2.\tImpact of fine-tuning the teacher network:\nThe paper adopts a static teacher network, which is obtained by fine-tuning SAM on labeled data and then freezing it. To validate the effect of fine-tuning, the paper should compare the performance of the original SAM and the fine-tuned SAM. \n\n3.\tClarification on the role of unlabeled data:\nThe role of unlabeled data in the framework needs further clarification. In Section 3.3, the paper states: “After fine-tuning, we further optimize the model through semi-supervised learning.” What does this “optimization” refer to exactly? Is the unlabeled data only used to help align the student’s predictions with those of the teacher?\n\n4.\tEvaluation of model deployment and storage optimization:\nAlthough the paper claims that the proposed model has less than 10% of the original SAM parameters, it does not provide a clear comparison of actual storage sizes (e.g., in MB or GB), nor does it report key deployment metrics such as inference speed or latency in clinical scenarios.\n\n5.\tDoes offline distillation limit the model's adaptability?\nThe teacher model is frozen after fine-tuning, while the student model continues to be updated during training. This offline distillation approach may prevent the teacher from adapting to the student’s evolving learning status or distribution shift. Has the author considered using a soft update strategy (e.g., EMA) to enhance the adaptability of the model?", "limitations": "Yes.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 3, "significance": 1, "originality": 2, "strengths_and_weaknesses": "The Related Work and Methodology are clearly written and easy to follow. However, the claimed innovation is questionable. The authors state that the core contribution lies in leveraging unlabeled data for semi-supervised learning. Actually, unlabeled data is only used for knowledge distillation and not for effectively improving SAM's adaptation to the target domain.\n\nOverall, the work does not constitute a true semi-supervised learning framework. The teacher network is fine-tuned using labeled data, while the student network is trained with hard labels and distills knowledge by aligning soft predictions on unlabeled data with those from the teacher. Unlabeled data serves merely as a medium for distillation, rather than directly enhancing the model’s generalization ability.\n\nMoreover, the paper lacks critical experiments, such as evaluating the performance of the teacher network itself and quantifying the performance gap between the teacher and student networks.", "was_revised": true, "final_justification": "The authors have not resolved my primary concern. Both the teacher and student models are trained on the same labeled data, and the student model additionally learns from the pseudo-labels and intermediate features generated by the teacher. I still do not understand why the student model would outperform the teacher.", "rebuttal": "# To Reviewer yrJD\n\nThank you for your thoughtful and constructive comment on our paper.\n\n## Weakness1: Core Contributions and Innovations.\n\nAnswer: The core contribution of the paper lies in exploring how to more effectively apply SAM to semi-supervised medical image segmentation tasks. To this end, we propose a novel fully SAM-based semi-supervised segmentation framework along with a tailored learning strategy. Utilizing unlabeled data for semi-supervised learning is a key component of the proposed strategy. \nSemi-supervised medical image segmentation aims to leverage a small amount of labeled data together with a large amount of unlabeled data for joint training, thereby fully mining the potential information in the unlabeled data. This approach significantly reduces dependence on expert annotations and improves the model's segmentation accuracy and generalization ability. Traditional semi-supervised methods usually rely on consistency regularization or pseudo-labeling mechanisms applied to the unlabeled data, whereas our method adopts a knowledge distillation strategy that enables the student model to effectively benefit from the unlabeled data. Therefore, essentially, our approach is semi-supervised learning.\nSpecifically:\n\n***(1)*** Our method combines two distillation strategies to supervise labeled data while controlling unlabeled data. This effectively transfers the teacher model's strong segmentation ability to the student model, significantly improving its adaptability to the target domain. And it is a common and effective strategy in semi-supervised medical image segmentation.\n\n***(2)*** In fact, without employing this strategy, effectively leveraging large amount of unlabeled data in the specialized task of semi-supervised medical image segmentation presents considerable challenges, and pseudo-labeling may be the only viable alternative. Notably, in our approach, after the teacher model is fine-tuned and fixed on labeled data, learning from unlabeled data essentially corresponds to a pseudo-label extraction process, although we employ an implicit encoder-decoder form. From this perspective, calculating consistency between the intermediate and output layers of the teacher and student models constitutes a supervised learning process, which constrains the student model's outputs using pseudo-labels (i.e., decoded features) generated by the fixed, powerful pre-trained teacher model.\n\n***(3)*** We input both unlabeled and labeled data into the model simultaneously to jointly train the student encoder-decoder. Since the quantity of unlabeled data far exceeds that of labeled data, its influence on the student model is non-negligible. Experimental results in the paper demonstrate the effectiveness of this framework for utilizing unlabeled data. For example, as shown in Figure 1 of the paper, taking the LA dataset as an instance, we validate the effectiveness of our method. The results indicate that when training the student SAM with only 5% labeled data, the average DSC reaches 79.09%. In contrast, by employing our method with 5% labeled data and 95% unlabeled data, the DSC significantly improves to 88.42%, achieving a gain of 9.33%. This clearly demonstrates that our method's utilization of unlabeled data substantially enhances the performance and generalization ability of the student model.\n\n***(4)*** In our method, supervision of unlabeled data mainly comes from the teacher model. Thus, we provide additional experiments to validate the teacher model's effectiveness (see Table 1).\n\n### Table 1: Comparison between original SAM and fine-tuned SAM using DSC as the evaluation metric. The latter was fine-tuned with 10% labeled data.\n\n| Dataset     | Original SAM | Fine-tuned SAM |\n|-------------|---------------|----------------|\n| LA          | 17.59         | 88.54          |\n| Brats2019   | 6.47          | 84.04          |\n| PROMISE12   | 2.53          | 85.39          |\n| Synapse     | 3.48          | 75.99          |\n\nTherefore, overall, our method is a semi-supervised medical segmentation approach. In our method, unlabeled data is not merely a medium for distillation but is deeply involved in training the student model, and effectively enhancing the student SAM's adaptation to the target domain.\n\n\n## Weakness2, Question1 and Question2: Effectiveness of the teacher SAM and comparison between teacher and student.\n\nAnswer: We acknowledge that the performance of the teacher model has a significant impact on the effectiveness of distillation, and we recognize the lack of quantitative analysis in the paper regarding the performance difference between the original SAM and the fine-tuned SAM. Although we mentioned in the paper that fine-tuning SAM can improve its performance on medical image segmentation tasks, specific experimental data were not provided. To further validate the effectiveness of the teacher model, we conducted comparative experiments on multiple datasets between the original SAM and the fine-tuned teacher SAM (see Table 1). The results demonstrate that the fine-tuned SAM significantly outperforms the original version across all datasets, confirming its effectiveness as the teacher model. Furthermore, we report the performance gap between the teacher and student models (see Table 2). Experimental results show that, benefiting from abundant unlabeled data and the proposed knowledge distillation-based learning strategy, the student model outperforms the teacher model on most datasets. Notably, our proposed hierarchical distillation pipeline extracts multi-level information to enhance the feature representation capability of the student model. Meanwhile, the dynamic loss weighting strategy gradually reduces the distillation intensity, decreasing the student's reliance on the teacher and partially mitigating the negative impact caused by insufficient teacher performance.\n\n### Table 2: Comparison between the fine-tuned teacher and the trained student using DSC as the evaluation metric.\n\n| Dataset       | Label Ratio | Teacher | Student |\n|---------------|-------------|---------|---------|\n| LA            | 5%          | 86.22   | 88.42   |\n|               | 10%         | 88.54   | 90.47   |\n| Brats2019     | 10%         | 84.04   | 85.14   |\n|               | 20%         | 85.73   | 86.46   |\n| PROMISE12     | 10%         | 85.39   | 83.41   |\n|               | 20%         | 86.00   | 86.44   |\n| Synapse       | 10%         | 75.99   | 78.42   |\n\n\n## Question3: Clarification on the role of unlabeled data.\n\nAnswer: The term \"further optimization\" in the paper refers to, based on the fine-tuned teacher model, utilizing unlabeled data together with labeled data to further improve the performance of the student model through semi-supervised learning. In this process, unlabeled data contains a wealth of valuable information that plays a crucial role in improving the overall performance of the student model. Specifically, we transfer the image embeddings and prediction outputs of the unlabeled data from the teacher model to the student model via knowledge distillation, enabling the student to learn richer representations during training. The role of unlabeled data can be further referenced in our response to Weakness1 above.\n\n\n## Question4: Evaluation of model deployment and storage optimization.\n\nAnswer: We place great emphasis on the feasibility of model deployment in practical scenarios. Therefore, in Table 3 of the supplementary material, we present the results of the model complexity comparison experiments. In Table 3, we compare the parameter size (Params) and computational complexity (MACs) between the teacher model based on the original SAM and the efficient student SAM used for inference. The results show that the student model significantly reduces model size and computational cost, with parameters decreasing from 105.05M to 9.51M and MACs from 103.40G to 10.33G, greatly improving inference efficiency. Additionally, Table 8 in the paper also presents a comparison of parameter size between our model and other medical SAM variants based on the original SAM design.\n\n\n## Question5: Offline distillation's limitation.\n\nAnswer: We indeed adopt an offline distillation strategy, wherein the original SAM (ViT-B) is first fine-tuned, then frozen and used as the teacher model to guide the student SAM (TinyViT) during training. Due to the significant architectural differences between the two networks, directly applying soft update strategies such as EMA (Exponential Moving Average) at the parameter level is infeasible, as the parameter dimensions and structures are incompatible and cannot be effectively aligned for updates.\nWe choose this offline distillation approach because the fine-tuned teacher already possesses strong feature representation capabilities. Meanwhile, we propose a dynamic loss weighting strategy to further mitigate potential mismatches between the fixed teacher and the continuously evolving student. This strategy automatically adjusts the weight of the distillation loss based on the student's learning status, enabling the student to rely more on the teacher's guidance in the early training stage and gradually transition to self-learning in later stages, thereby enhancing the overall effectiveness of the distillation process. In future work, we are also interested in exploring improved learning strategies to further enhance the flexibility of knowledge distillation and the adaptability of the model."}, {"review_id": "QiMS2OD1LH", "reviewer": "Reviewer_kvmC", "summary": "This paper presents a novel fully SAM-based semi-supervised medical image segmentation (SSMIS) framework designed to address the scarcity of annotated medical imaging data. It tackles the suboptimal performance of generalist SAM models on medical images by integrating an efficient SAM variant as the backbone and updating its default prompt embedding.", "questions": "1. Could the authors provide a more granular breakdown of the \"various medical datasets\" used, specifying modalities, anatomical regions, and the range of pathological conditions to better assess the true generalizability?\n2. How sensitive is the model's performance to the choice of distillation strategy parameters and the dynamic loss weighting schedule? Is there an automated way to tune these, or does it require significant manual effort per dataset?", "limitations": "1. **teacher-student model dependency:** The fundamental effectiveness of the proposed semi-supervised framework relies on the \"teacher\" (original SAM) providing accurate and rich knowledge. If the teacher struggles with the inherent complexities, ambiguities, or out-of-distribution characteristics of certain medical images, the student's performance will be capped, potentially leading to critical errors in clinical diagnoses or interventions.\n2. **generalizability to highly fiverse medical modalities/anatomies:** Medical imaging is incredibly diverse. A model performing well on one type of medical data (e.g., cardiac MRI) may not directly generalize to another (e.g., lung CT scans with diffuse pathologies) due to different image characteristics and anatomical contexts. This limits the \"plug-and-play\" nature of the solution for broad clinical applicability and necessitates careful validation for each new application.\n3. **complexity of hierarchical knowledge transfer and dynamic weighting:** The intricate distillation and weighting mechanisms, while contributing to performance, increase the complexity of model development and deployment. Achieving optimal results likely requires expert-level hyperparameter tuning, which could be a barrier for wider adoption in clinical research or practice where ease of use is paramount.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n1. The paper proposes a novel framework that fully leverages an efficient SAM variant as the backbone for semi-supervised medical image segmentation.\n2. The sophisticated knowledge distillation strategy from an original SAM teacher to optimize the efficient student SAM backbone is interesting.\n3. It updates SAM's default prompt embedding, and mitigate the negative impact of inaccurate prompts.\n\n**Weakness:**\n1. The student model's performance is inherently tied to the capabilities of the original SAM teacher. If the teacher, even after fine-tuning on medical data, struggles with certain rare pathologies or complex anatomical variations, these limitations will be propagated, potentially capping the student's accuracy in critical cases.\n2. While \"various medical datasets\" are mentioned, the specific range of medical imaging modalities (e.g., MRI, CT, X-ray, ultrasound, different anatomical regions, varying image qualities) and their pathological diversity are not explicitly detailed in the abstract. SAM's generalization can still be challenging across wildly different medical image characteristics.", "was_revised": true, "final_justification": "Novel and interesting work. I keep my original score.", "rebuttal": "# To Reviewer kvmC\n\nThank you for your thoughtful and constructive comment on our paper.\n\n## Weakness1, Limitation1 and Limitation2: Teacher-student model dependency and SAM's limitations in complex medical scenarios.\n\nAnswer: We agree that the performance of the student model can be influenced by the effectiveness of the teacher model. In particular, when dealing with rare pathologies or complex anatomical structures in medical images, the teacher SAM may still face challenges, even after fine-tuning. We have stated these limitations in the Supplementary Material (Section 4) in accordance with the checklist requirements. \nIt is important to emphasize that the primary focus of this work is not to enhance SAM's performance in complex medical scenarios, but rather to build an effective and efficient semi-supervised medical image segmentation framework based on it. Perhaps SAM has limitations in extreme cases, but replacing it with a stronger medical-specific model may further improve the performance of our proposed framework.\nMoreover, our proposed knowledge distillation strategy helps mitigate the negative impact of imperfect teacher predictions (see Table 1). Specifically, the proposed hierarchical distillation pipeline enables the student model to benefit from multi-level feature transfer, promoting more robust feature representations. In addition, the dynamic loss weighting mechanism gradually reduces the distillation intensity during training, thereby alleviating over-reliance on the teacher and enhancing the student's robustness when encountering challenging or rare cases.\nTherefore, although the SAM teacher may introduce some risk of error propagation in extreme scenarios, our method focuses on how to make effective use of powerful foundational models like SAM under practical constraints. We believe this direction is of significant value for advancing semi-supervised medical image segmentation paradigms. In future work, we plan to further explore strategies to reduce the impact of teacher errors on student model optimization.\n\n### Table 1: Comparison between the fine-tuned teacher and the trained student using DSC as the evaluation metric.\n\n| Dataset       | Label Ratio | Teacher | Student |\n|---------------|-------------|---------|---------|\n| LA            | 5%          | 86.22   | 88.42   |\n|               | 10%         | 88.54   | 90.47   |\n| Brats2019     | 10%         | 84.04   | 85.14   |\n|               | 20%         | 85.73   | 86.46   |\n| PROMISE12     | 10%         | 85.39   | 83.41   |\n|               | 20%         | 86.00   | 86.44   |\n| Synapse       | 10%         | 75.99   | 78.42   |\n\n\n## Weakness2 and Question1: Description of various medical datasets and detailed fine-grained segmentation results.\n\nAnswer: Admittedly, we did not elaborate on the \"various medical datasets\" used in the abstract. However, the datasets are clearly listed in the Experiments section and further detailed in the Supplementary Material (Section 1). These datasets cover a wide range of imaging modalities, anatomical regions, and pathological conditions, including: contrast-enhanced MRI for left atrial segmentation (LA dataset), multi-modal MRI for brain glioma segmentation (BraTS-2019, using only T2-FLAIR), MRI for prostate segmentation (PROMISE12), and CT for multi-organ abdominal segmentation (Synapse). The Synapse dataset involves eight abdominal organs: aorta, gallbladder, spleen, left kidney, right kidney, liver, pancreas, and stomach. \nThe first three datasets focus on single organs such as the left atrium, brain glioma, and prostate, demonstrating our method's effectiveness on these tasks. The fourth dataset is a multi-organ abdominal segmentation dataset, so we provide detailed fine-grained segmentation results for each organ in Table 8 of the paper.\nIn addition, to verify the generalizability of our method, we introduced a new chest X-ray dataset called CXRS. This dataset contains 1,254 high-resolution chest X-ray images, each annotated at the pixel level with 31 anatomical structures, including 24 ribs, 2 clavicles, 2 scapulae, 2 lungs, and 1 mediastinum. The dataset is split into training, validation, and test sets in a 7:1:2 ratio, comprising 879, 125, and 250 images respectively. Experimental results are shown in Table 2, demonstrating that our method also achieves promising performance on this dataset.\n\n### Table 2: Comparisons with some semi-supervised segmentation methods on the CXRS dataset with 10% labeled data.\n\n| Method   | DSC ↑  | 95HD ↓ |\n|----------|--------|--------|\n| UA-MT    | 72.14  | 23.69  |\n| DTC      | 75.31  | 17.10  |\n| BCP      | 78.79  | 10.61  |\n| Ours     | 79.45  | 10.32  |\n\nBased on the above studies, our datasets cover three different imaging modalities: MRI, CT, and X-ray. They include a variety of diseases and involve anatomical regions such as the heart, brain, prostate, abdomen, and chest, as well as various detailed subregions. This diversity and representativeness fully support a comprehensive evaluation of the generalizability of our method.\n\n\n## Question2: Distillation strategy parameters and the dynamic loss weighting schedule.\n\nAnswer: Thanks for your reminder and suggestion. We also attach great importance to the configuration setting of the distillation pipeline and the dynamic loss weighting strategy. As shown in Table 4 of the paper and Section 3 of the supplementary material, we conduct an ablation study on the key parameters of the distillation strategy by exploring the loss weight ratio between embedding distillation and output distillation on the LA and PROMISE12 datasets. Based on the results, we adopt the configuration of $\\lambda_{emb}$ : $\\lambda_{out}$ = 1/3 : 2/3, which also demonstrates stable performance across other datasets. For the dynamic loss weighting strategy, we compare the losses of the teacher and student models and attenuate the distillation intensity only when certain conditions are met [1]. The scaling factor $\\alpha$= 0.95 is empirically set and validated through extensive experiments, consistently yielding robust results across different datasets. Therefore, we do not introduce additional automatic tuning mechanisms in this work, nor do we require extensive manual tuning to achieve strong performance, highlighting the generalizability and stability of our method.\n\n\n## Limitation3: Complexity of hierarchical knowledge transfer and dynamic weighting.\n\nAnswer: We acknowledge that introducing hierarchical knowledge distillation and dynamic loss weighting mechanisms increases the model complexity to some extent. However, our method essentially consists of a teacher-student framework, two simple distillation branches, and a dynamic loss weighting mechanism. It involves only a small number of hyperparameters and does not require expert-level tuning. Ablation studies of the paper further demonstrate that these hyperparameters are controllable and easy to adjust. Moreover, our model development was primarily conducted on a single NVIDIA RTX 3090 GPU (using two RTX 3090 GPUs for the Synapse dataset), indicating low computational resource requirements. \nOverall, our method is easy to use and easy to generalize, which facilitates its adoption in clinical research and practice.\n\n\n[1] Huang Y, Wang Y, Tai Y, et al. Curricularface: adaptive curriculum learning loss for deep face recognition[C]//proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 5901-5910."}]

Author rebuttal:

Meta-review: Authors present a semi-supervised learning method for medical image
segmentation. The main novelty is to use SAM to this end. In an
elegant approach, authors first fine-tune two SAM models. The first -
bigger one - is updated with low parameter updates, while the second -
the smaller one - is updated completely. After fine-tuning,
semi-supervised learning happens in a teacher-student setup. However,
in their setup, authors use distillation and freeze the teacher
network - the bigger SAM. Distillation works both on the output but
also on the embedding space.

Experiments with three important benchmark datasets are
presented. Results are quite promising for the proposed
approach. Ablation studies are also conducted. They show the imprtance
of fine-tuning of the student model as well as using two losses during
knowledge distillation. The contribution of the dynamic loss weighting
remains minimal, however, the presence of this experiment is valuable.

The main concerns are related to the level of novelty and the fact
that the student and teacher models are fine-tuned on the same
dataset. However, despite these concerns, three out of four reviewers
praise the contribution and acknowledge that this would lead to
further work and interesting discussions at the conference. One
reviewer remains rather negative about the article, however, I do not
share the concerns of this reviewer about the article.

**Summary**: I find this article to be a solid contribution. While the
  novelty in terms of technical components may not be very high, the
  approach is still novel for semi-supervised learning, as it utilizes
  SAM models. Furthermore, results on three benchmark datasets are
  quite promising. I believe this article will lead to further
  discussions and interesting future work.
Decision: Accept (poster)