Target and task specific source-free domain adaptive image segmentation

Authors: vibashan  vs
jeya  maria
jose  valanarasu
vishal m patel
Venue: MIDL_io_2024_Conference
Published: 2024-01-06 20:49:27

Abstract: Solving the domain shift problem during inference is essential in medical imaging as most deep-learning based solutions suffer from it. In practice, domain shifts are tackled by performing Unsupervised Domain Adaptation (UDA), where a model is adapted to an unlabeled target domain by leveraging the labelled source domain. In medical scenarios, the data comes with huge privacy concerns making it difficult to apply standard UDA techniques. Hence, a closer clinical setting is Source-Free UDA (SFUDA), where we have access to source trained model but not the source data during adaptation. Methods trying to solve SFUDA typically address the domain shift using pseudo-label based self-training techniques. However due to domain shift, these pseudo-labels are usually of high entropy and denoising them still does not make them perfect labels to supervise the model. Therefore, adapting the source model with noisy pseudo labels reduces its segmentation capability while addressing the domain shift. To this end, we propose a two-stage approach for source-free domain adaptive image segmentation: 1) Target-specific adaptation followed by 2) Task-specific adaptation. In the Stage-I, we focus on learning target-specific representation and generating high-quality pseudo labels by leveraging a proposed ensemble entropy minimization loss and selective voting strategy. In Stage II, we focus on improving segmentation performance by utilizing teacher-student self-training and augmentation-guided consistency loss, leveraging the pseudo labels obtained from Stage I. We evaluate our proposed method on both 2D fundus datasets and 3D MRI volumes across 7 different domain shifts where we achieve better performance than recent UDA and SF-UDA methods for medical image segmentation. Code is available at https://github.com/Vibashan/tt-sfuda.

Main paper: 
 introduction: 
 Deep-learning based methods such as convolutional neural networks (Ronneberger et al., 2015; Zhou et al., 2018; C ¸içek et al., 2016; Milletari et al., 2016; Islam et al., 2018; Valanarasu et al., 2020) and transformers (Chen et al., 2021b; Valanarasu et al., 2021; Hatamizadeh et al., 2022) have been the leading solutions for medical image segmentation. One major concern with deep neural networks (DNNs) is that they depend very much on the dataset they are trained on. DNNs trained on a particular dataset do not perform well when tested on a different dataset even if it is of the same modality. Small changes in camera type, calibration properties, age and demographics of patients causes a distribution shift which results in a considerable drop in performance. It is very common for a domain shift to exist during deployment of medical imaging solutions in real-time (Vashist, 2017) . Hence adapting a source model to a target distribution is an important problem to solve for medical imaging applications like segmentation.Recently, many works have explored unsupervised domain adaptation (UDA) for medical image segmentation (Javanmardi and Tasdizen, 2018; Panfilov et al., 2019; Zhang et al., 2019; Yang et al., 2019; Perone et al., 2019) . However, UDA methods work under the assumption that labeled source domain data and unlabeled target domain data are available during adaptation. This assumption often does not hold, particularly in medical applications, where data sensitivity and privacy concerns are crucial. Consequently, utilizing source data during adaptation, given its availability concerns, becomes problematic. Source-Free Unsupervised Domain Adaptation (SFUDA) is a established solution where we assume we have access only to the source-trained model and target data for adaptation. This setting offers a practical approach to clinical adaptation without relying on source data. Bateson et al. (Bateson et al., 2020) proposed an SFUDA method that reduces label-free entropy loss for target-domain data. Similarly, Chen et al. (Chen et al., 2021a ) introduced a denoised pseudo-label technique for self-adaptation on the target data. Although these techniques facilitate model adaptation to the target domain, they often compromise the model's taskspecific capabilities, such as segmentation performance, due to noisy pseudo-labels.In this work, we propose a novel method focusing on adapting the source model to the Target distribution while also making sure the Task-oriented representation is preserved for Source-Free Unsupervised Domain Adaptation (TT-SFUDA). Specifically, we introduce a two-stage pseudo-label based adaptation framework: Stage I -Target-specific adaptation and Stage II -Task-specific adaptation. In target-specific adaptation, we aim to learn targetspecific representation to adapt to the target domain while generating high-quality PL. In task-specific adaptation, we aim to improve task-related (i.e segmentation) performance on target domain by leverage high-quality PL generated from Stage-I. To achieve this in Stage-I, we propose an ensemble entropy minimization loss and selective voting strategy to learn target-specific representation and generate high-quality pseudo-labels, respectively. In Stage -II, we employ a strong-weak augmentation based teacher-student self-training framework and agumentation-based consistency loss to improve the segmentation performance target domain by leveraging high-quality PL generated from Stage-I. We validate our method for many domain shifts for both 2D and 3D medical datasets and compare with recent SFUDA and UDA adaptation methods.In summary, this work makes the following contributions: 1) We propose a new SFUDA method focusing on both target and task specific adaptation for image segmentation. 2) We introduce an ensemble entropy minimization loss and a selective voting strategy to learn target-specific representation and generate high-quality pseudo labels. 3) We propose a teacher-student self-training framework with augmentation-based consistency to improve segmentation performance on the target domain. 4) We conduct extensive experiments on both 2D and 3D data of different modalities. We show results on 7 different data-shifts for both 2D fundus images and 3D MRI volumes and outperforms recent methods.TT-SFUDA 
method: 
 Our proposed TT-SFUDA method for medical image segmentation consists of two parts: Stage I -Target specific adaptation and Stage II -Task specific adaptation. First, we explain the base segmentation network architecture and formulate certain preliminaries and notations used in the proposed adaptation method. Base Network Details: We use a regular UNet (Ronneberger et al., 2015) as the base segmentation network. We use a 5 level encoder-decoder framework. Each conv block in the encoder consists of a conv layer, ReLU activation and a max-pooling layer. Each conv block in the decoder consists of a conv layer, ReLU activation and an upsampling layer. For upsampling, we use bilinear interpolation. For our experiments on 3D volumes, we use a 3D UNet architecture (C ¸içek et al., 2016) with same setup replacing 2D conv layer with 3D conv layers, 2D max-pooling with 3D max-pooling and bilinear upsampling with trilinear upsampling. The segmentation loss used to train this network L seg is as follows:EQUATIONNotations: We denote the input data as x and labels as y. We denote the source domain distribution as D s = {(x n s , y n s )} Ns n=1 and target domain distribution asD t = {(x n t , y n t )} Nt n=1where s represents source, r represents target, N s and N t denotes number of data-instances in source and target data respectively. We denote the source model of UNet as Θ s . The goal is to adapt Θ s from D s to D t while using only the target data x t . We assume we do not have access to y t which makes it unsupervised and also assume we do not have access to D s which makes it source-free.First, we address the domain shift which is the main problem that leads to the drop in performance. To perform this, we use the adaptation framework below: Adaptation Framework: We define two instances of the model: the source model (Θ s ) and the target model (Θ stage1 t). The source model, Θ s , serves as a pseudo model for generating pseudo labels. A straightforward approach to learning target-specific representations is to train the Θ stage1 t model using pseudo-labels ŷn t generated from Θ s . However, this method is highly dependent on the quality of the pseudo-labels. Due to the domain shift, these pseudo-labels are typically noisy, and training the model with noisy pseudo-labels can hinder the acquisition of task-specific information, usually resulting in decreased performance. Thus, in target-specific adaptation, our initial focus is on reducing the entropy of the model to produce reliable pseudo-labels, which are then utilized to enhance task-specific representation learning. Therefore, the source model Θ s is initialized with weights from a model trained on the source domain and is kept frozen, while the target model (Θ stage1 t ) gradually adapts to the target domain by minimizing the Stage-I loss.For entropy minimization, we can generate an entropy map H(ŷ n t ) using output prediction map ŷn t by:EQUATIONMinimizing H(ŷ n t ) will result in overall minimization of model entropy. However, using just one input data to generate the entropy map might not give us all the accurate locations of high entropy. Therefore, to improve the overall entropy suppressing property of model Θ stage1 t , we propose an ensemble entropy minimization approach. For this, we first generate a set of augmented data Xn t = {aug i (x n t )} M i=1 where M is the number of augmentations used. We then generate a set of pseudo labels Ỹ n t corresponding to these augmented inputs Xn t using the source model (Θ s ). Note that Xn t and Ỹ n t denotes a set of data instances {x n t } M i=1 and {y n t } M i=1 depending on the number of augmentations used. Based on the generated pseudo labels Ỹ n t , we generate entropy maps H( Ỹ n t ) as follows:EQUATIONNow, we define the ensemble entropy minimization loss L EEM as follows:EQUATION)This loss not only tries to suppress the entropy of the original pseudo-label but also of the pseudo-labels of the augmented ones making it more generalized. Enhancing Pseudo-Label by Selective Voting: Minimizing entropy is an effective strategy for adapting the model to the target domain. Furthermore, improving the quality of pseudo-labels can lead to better adaptation, as it provides direct supervision to Θ stage1 t . To enhance the accuracy of these pseudo-labels, which are often noisy, we propose reducing the false-negative areas within the pseudo-labels, which are typically associated with regions of high entropy. This is achieved by using the entropy maps H(ŷ n t ) and pseudo-labels Ỹ n t , generated from augmented inputs as discussed in the preceding section. Initially, we utilize Ỹ n t to identify entropy maps that exceed a specified threshold δ. This step ensures the preservation of high-entropy regions across different augmentations, highlighting predictions near the class boundary. Subsequently, we create a normalized entropy map by integrating the entropy map derived from non-augmented inputs H(ŷ n t ) with the selected entropy maps from augmented inputs H( Ỹ n t ).EQUATIONFrom H S , we generate the selective entropy mask (Z n t ) by thresholding H S with 0.5 and we then generate a false negative mask (U n t ) as follows:EQUATIONIn practice, we find that the optimal values for λ 1 and λ 2 is between 0.25 and 0.5. Finally, we suppress the false negatives in U n t by performing AN D operation with the selective entropy mask (Z n t ) and obtain enhanced pseudo-label ȳn t . We visualize the intermediate steps for a sample input x n t in Fig. 4 . It can be seen that the selectively voted entropy map H S captures more consistent high entropy regions compared to the normal entropy map H(ỹ n t ). Our enhanced pseudo-label ȳn t can also be observed to be a refined version of the original pseudo-label ŷn t . To train Θ stage1 t , we define the stage-I training loss L stage1 as:L stage1 = L seg (x n t , ȳn t ) + L EEM (ŷ n t ). (7)Although the model weights have been adapted for the new target data distribution, they do lose some task-specific information as the pseudo-labels are not perfect. Thus, targetspecific adaptation to the new domain causes some unwanted hindrance to the network's segmentation capability due to supervising on imperfect pseudo-labels. To solve this, we further refine the model in this stage to improve its task-specific learning capabilities.Teacher-Student Self-Training: We use a Teacher-Student model (Wang and Yoon, 2021) where the student model Θ student t is optimized using pseudo-labels that are generated from the teacher model Θ teacher t . Note that both the teacher and student are initialized with the model obtained from Stage I i.e Θ stage1 t . This makes sure that the pseudo-labels generated are already of less uncertainty as Θ stage1 t has been trained previously to minimize entropy. The teacher model is updated by gradually transferring the weights of the continually learned student model. This is done by using exponential moving average (EMA) to update the teacher model. Thus, the teacher model can be considered as an ensemble of different student models from various training iterations. Augmented-guided Consistency: To make sure that the segmentation capabilities of the model is improved, we use an augmentation-guided consistency loss while doing the teacher-student self-training. First, we perform some strong augmentations on target input data x n t to obtain xn t . We also perform some weak augmentations on x n t to obtain xn t . xn t is then passed through the teacher model to obtain ỹn t . xn t is passed through the student model to obtain ŷn t . As the pseudo-labels generated for both these augmentations need to be consistent, we define the augmentation consistency loss L AC as follows:EQUATION)Note that Θ teacher latent corresponds to the features extracted from the latent space of Θ teacher model. We extract features from the last layer of the encoder while calculating the loss L AC . The total loss for stage II L stage2 along with the pseudo-label supervision is defined as follows:L stage2 = L seg (x n t , y n t ) + L AC (x n t , xn t ). ( 9)The final adapted model weights of stage II is represented as Θ stage2 t is basically Θ teacher t after the self-training process. 
experiments and results: 
 Datasets: For 2D experiments, we focus on the task of retinal vessel segmentation from fundus images. We make use of the following datasets: CHASE (Fraz et al., 2012) , RITE (Hu et al., 2013) and HRF (Odstrčilík et al., 2009) .For 3D experiments, we focus on brain tumor segmentation from MRI volumes. We make use of the BraTS 2019 dataset (Menze et al., 2014; Bakas et al., 2017 Bakas et al., , 2018) ) which consists of four modalities of MRI-FLAIR, T1, T1ce and T2. We study the domain shift problems between these four modalities for volumetric segmentation of brain tumor. This is a multi-class segmentation problem with 4 labels. We randomly split the dataset into 266 for training and 69 for validation. Implementation Details: For 2D source experiments, we train UNet using L seg (x s , y s ) with an Adam optimizer and a learning rate of 0.001, momentum of 0.9. We also use a cosine annealing learning rate scheduler with a minimum learning rate upto 0.0001. For 3D experiments, we use a similar loss but is for multi-class segmentation. In TT-SFUDA experiments, for both stages, we use the Adam optimizer with a learning rate of 0.0001 and momentum of 0.9. For Stage I, we use color jitter, grayscale, color contrast etc. as different augmentation and we set hyperparameters α, λ 1 and λ 2 to 0.75, 0.3 and 0.5 respectively. For stage II, the teacher EMA rate is set to 0.99 for each iteration in the student-teacher network. We train Stage I for one epoch and Stage II for ten epochs (Refer to supplementary for more details). The framework is implemented with Pytorch using NVIDIA TitanXp GPU.We compare our proposed method TT-SFUDA against recent methods proposed for both UDA and SFUDA. Note that UDA methods have an advantage as they have access to the source train data to help the adaptation process. For UDA, we compare against a normal pseudo-label self-training (Zou et al., 2018) based approach; BEAL (Zou et al., 2018) which uses adversarial learning between source and target data with boundary information specifically for fundus images; Output-Space (Tsai et al., 2018) which involves output space projection consistency for domain adaptive segmentation and AdvEnt (Vu et al., 2019) which ensures entropy consistency between the source and target domains. For SF-UDA, we compare against SRDA (Bateson et al., 2020) and DPL (Chen et al., 2021a) . SRDA uses a task prior while DPL uses an uncertainty-guided denoised pseudo-label based approach. Note that we adopt these methods for 3D by making necessary changes. We leave out BEAL and Outer-Space for 3D comparison as they have been specifically designed for 2D. More details about this can be found in supplementary material. In Tables 1 and 2 , we tabulate our results for domain shifts in 2D fundus images and 3D MRI volumes respectively. We achieve better performance compared to both UDA and SFUDA baselines across almost all shifts. We present sample qualitative results in Fig. 3 where we observe a good performance for TT-SFUDA when compared to other methods. Ablation Study: Table 3 presents the results of different combinations of stage-wise experiments on 2D domain shifts. The first two rows correspond to individual stage adaptation performances. The last two rows correspond to different sequences of stage-wise adaptation performances. From the first two rows, we can observe that Stage II is performs better than Stage I when there is a huge gap between direct testing and oracle performance. Performing task-specific adaptation makes the model focus more on segmentation rather than learning target-specific representation which essentially leads to noise overfitting. Therefore, when there is a large domain shift (CHASE→HRF), Stage II has more room for task-specific improvements but fails to learn target-specific representation in case of a small domain shift (CHASE→RITE). This is clearly observed in Stage II → Stage I experiment, where first learning task-specific representation essentially overfits the model to the noise generated from the pseudo-labels. Consequently, when we perform entropy minimization for targetspecific adaptation, the model performance drops by a large margin. Moreover, we can observe that the drop in Stage II → Stage I is directly proportional to the difference in the domain gap. Therefore, target-specific adaptation followed by task-specific adaptation produces best performance as seen from the Stage I → Stage II experiment. 
conclusion: 
 In this work, we proposed a new method for source-free unsupervised domain adaptive image segmentation called TT-SFUDA. We perform TT-SFUDA in two stages: Stage I-Target specific adaptation and Stage II -Task specific adaptation. In the first stage, we learn targetspecific representation and generate high-quality pseudo-labels. In the second stage, improve the segmentation performance by leveraging a self-training framework with augmentationguided consistency loss. Evaluation on multiple 2D and 3D domain shifts datasets show the effectiveness of our method compared to recent UDA and SFUDA methods. 
 
 Appendix:  


Decision: This paper proposes a source-free unsupervised domain adaptation method for segmentation using a 2 stage approach: target-specific adaptation followed by task-specific adaptation. 

Strengths:
+ UDA is important research direction, and source-free is particularly relevant for medical imaging domain
+ Novel two-stage approach for handling the problem
+ Good experiments with multiple datasets and ablation studies

Weaknesses:
- No validation data used for hyper parameter tuning 
- While standard deviations were given, significance testing was not performed, and there are differing opinions regarding whether the results show significant increase in performance compared to other SOTA

Following rebuttal, the reviewers felt that the strengths generally outweighed the weaknesses. While there were some concerns regarding experimental results, overall I think this paper would be of high interest to the MIDL community, and therefore recommend accept.

Reviews: review: paper_summary: paper_summary: The authors proposed a new SFUDA method for domain adaptation without accessing source data, which is useful in medical imaging field due to data privacy. The study focused on not only domain adaption to the target dataset but also image segmentation-specific adaptation by introducing entropy minimization and selective voting for better pseudo-labels. They also incorporated the teacher-student model for better tuning to help increase model performance.
main_review: main_review: The authors have addressed the questions raised in previous comments: (1) provided the descriptive statistics of the dataset (2) explained why there is no validation dataset (3) explained why the domain adaptation is not optimal in all cases. Overall, the authors provided a novel approach for domain adaptation in target specification and task specification and evaluated the approach in multiple datasets of different modalities.
strength_weakness: strength_weakness: 1. Overall, the paper is well-written, but there are certain areas where a bit more clarification would enhance understanding.
2. There is no validation set for the hyper-parameter tuning. This will increase reliability as the authors could choose the best-performing model in the validation set to compare across different methods.
3. In Table 2, the proposed method outperforms most of the other methods, but in FLAIR to T2 domain shift, it does not perform the best. The authors could discuss more and give insights on why this is the case.
questions: questions: 1. The authors performed a comprehensive ablation study and delivered a complete discussion on the difference in the performance. However, there are several domain shifts scenario that the proposed method did not perform better than the other methods (Table 2) and the authors did not further explain why this happens or the factor they believe that might play a role in this. For example, is it because of the inherent propagation of variance of the proposed method framework since there are many "moving parts" in the pipeline? Or is it because of the challenge of the specific type of domain shift that the authors are doing (e.g. FLAIR to T2 domain adaptation is generally hard and all methods have already reached the limit so that the difference in the performance do not have statistical significance)?
paper_summary: paper_summary: This paper proposes a two-stage approach: 1) Target-specific adaptation and 2) Task-specific adaptation. Stage-I improves pseudo-label quality with an entropy minimization loss and selective voting. Stage-II enhances segmentation using teacher-student training and consistency loss. Our method outperforms recent UDA and SF-UDA for medical image segmentation.
main_review: main_review: The study did not show a substantial improvement in results for 2D domain shifts, suggesting the need for further validation. Incorporating statistical significance into the results would offer a more robust assessment of the outcomes, strengthening the findings. Additionally, performing bootstrapping on the test results could enhance their reliability and generalizability. To expand the scope and applicability of the proposed method, it would be beneficial to test it on chest X-rays (2D) or CT images, in addition to the fundus images (2D) that were already evaluated. This broader testing could provide insights into the method's performance across different types of medical imaging data.
strength_weakness: strength_weakness: - The study did not demonstrate a significant enhancement in results for 2D domain shifts. 
- To strengthen the findings, it is recommended to incorporate statistical significance into the results, providing a more robust assessment of the outcomes.
questions: questions: Could be nice to try the proposed method for testing with chest X-rays (2D) or CT images, in addition to the tested fundus images (2D).
paper_summary: paper_summary: In this paper, the authors present a novel method for unsupervised domain adaptation in medical image segmentation task. The research question is of great importance for the medical image analysis community considering large amounts of unlabeled data. The experimental results are convincing in showing the presented model at least being strongly competitive with SOTA frameworks under the designed scenarios.
main_review: main_review: I would like to thank the authors for providing comprehensive rebuttals answering all my concerns. Based on the clarifications and additional information, I am more than willing to improve my rating for this paper.
strength_weakness: strength_weakness: 1. In stage I, the inclusion of original pseudo label entropy in loss is confusing.
2. The description to the selective voting idea is clear, but lacking experimental results to back up its effectiveness. 
3. The same goes with the $L_{ac}$ loss in the second stage. I understand this design as aiming to generalize latent space representation under diverse augmentations. But penalizing latent space may also cause some numerical problems in training. Please consider adding evidence to support the design choice.
4. The framework relies on another segmentation model with reasonable performance to start. If initial segmentation fail significantly, the framework may fail to detect and get trained on misleading pseudo labels.
questions: questions: 1. In section 2.2, the authors argue that the loss function jointly suppress entropy in original and augment pseudo labels. Considering the model weights of original model is fixed during training, how is the loss going to suppress entropy in labels? What is the point of including a fixed-weight model output into the loss function.

2. Please consider adding evidence for the effectiveness for selective voting and $L_{ac}$. Also, how difficult it is to tune the threshold parameter $\lambda_1, \lambda_2$ in your experiments?

Comments: title: Official Reply to Reviewer rTyX
Official Reply to Reviewer 5jne
Official Reply to Reviewer JEQu
comment: ***1) Stage I inclusion of pseudo-label entropy:***

In the SFUDA, we adapt a source-trained model without access to the original source data and due to domain shifts, initial pseudo-label predictions often exhibit high uncertainty. In response, Stage I of our method prioritizes the generation of reliable pseudo-labels for the subsequent Stage II, which involves student-teacher training. To enhance the reliability of these pseudo-labels, we specifically aim to reduce their inherent uncertainty. This is achieved by applying various data augmentations, which serve to expand the target data, and by systematically minimizing the entropy across all corresponding predictions. Overall, the need for original pseudo-label entropy is designed to minimize the uncertainty associated with our pseudo-labels from the source-trained model, enabling a reliable pseudo-label generation process.

***2) Subscript is misleading:***

Thank you for your suggestion. We will update it in our final version to avoid confusion.

***3) Strong and weak augmentation:***

For strong augmentation, we perform ColorJitter, RandomGrayscale, RandomSolarize, and RandomAutocontrast. For weak augmentation, we just perform Randomflip. We will update this detail in our final version.

***4) Section 2.2 Frozen weight and loss update:***

Thank you for pointing out the confusion in Section 2.2  and we apologize for the misunderstanding. As illustrated in Figure 1, we have two models: the source-trained model (frozen) and the Stage-I model (trainable). Since we don't have access to the source data, we always maintain a frozen source-trained model to ensure the preservation of the source domain information. The Stage-I model is initialized with weights from the source-trained model and gradually adapts to the target domain by minimizing the Stage-I loss. Therefore, the loss is backpropagated only to the Stage-I model, while the source-trained model remains frozen throughout the training process. We will clarify this detail and update Section 2.2 in the revised version.

***5) Penalizing latent space evidence and motivation:***

The loss in the latent space is motivated by self-supervised methods like SimCLR[1], BYOL[2], and MoCo[3]. These approaches demonstrate that models learn robust features when they produce similar representations for different augmentations of the same image by minimizing the loss at latent space. Applying this principle, our consistency loss aims to enhance the segmentation ability of both teacher and student models by ensuring they extract comparable and robust features from varied augmented inputs. This method is grounded in proven self-supervised learning techniques that improve model generalization and robustness.

***7) Parameter senstive:***

For both Stage I and Stage II, all loss weights were set to 1. Furthermore, our experiments on the variation in the $\lambda_1 - \lambda_2$ range had a negligible impact on performance, specifically within $\pm$ 1%. Overall, this indicates that while selective voting can effectively reduce false negatives, fine-tuning of the threshold parameter, especially within this critical range, is nuanced but not overly sensitive. We will clarify these detail in our revised version.

***8) Initial segmentation fail:***

Thank you for your insightful comment. I agree that our framework, like other Source-Free Unsupervised Domain Adaptation (SF-UDA) methods, depends on a reliable initial segmentation model. If the initial model is weak or the domain gap is large, our approach could struggle, similar to other SF-UDA methods. This issue is a common challenge in SF-UDA methods due to their dependency on pre-trained models and assumptions regarding domain similarity [4]. We acknowledge this limitation and believe that exploring ways to enhance model robustness against large domain discrepancies under reasonable settings is a promising direction.


***Reference:***

1. Chen, Ting, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. "A simple framework for contrastive learning of visual representations." In International conference on machine learning, pp. 1597-1607. PMLR, 2020.

2. Grill, Jean-Bastien, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch et al. "Bootstrap your own latent-a new approach to self-supervised learning." Advances in neural information processing systems 33 (2020): 21271-21284.

3. He, Kaiming, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. "Momentum contrast for unsupervised visual representation learning." In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9729-9738. 2020.

4. Mehra, Akshay, Bhavya Kailkhura, Pin-Yu Chen, and Jihun Hamm. "Understanding the limits of unsupervised domain adaptation via data poisoning." Advances in Neural Information Processing Systems 34 (2021): 17347-17359.
***1) More statistical significance for results:***

We appreciate the reviewer's suggestion. Regarding statistical significance, the standard deviations associated with our method across all domain shifts are consistently lower or comparable to those of the best-performing methods. This suggests that TT-SFUDA is not only effective but also stable. Furthermore, it is noteworthy that, unlike typical UDA methods which assume access to both source and target data, our method assumes access only to source-trained data, making it challenging. Nonetheless, the standard deviation is lower than that of most UDA methods, such as BEAL and Advent. Specifically, our TT-SFUDA method significantly outperforms others, with notable improvements in the dice score from 56.32 ± 1.23 (DPL, the second-best in SFUDA) to 58.25 ± 1.06 (TT-SFUDA) in the CHASE→HRF shift, and from 56.92 ± 1.99 to 58.37 ± 1.03 in the HRF→RITE shift, indicating superior performance with less variability. We include these details in the final version of our paper. 

***2) 2D chest X-rays or CT images:***

Thank you for your suggestion. We acknowledge the potential value of testing with chest X-rays or CT images. However, the fundus datasets such as HRF, CHASE_DB1, and RITE present unique challenges due to their extremely small size, comprising 18, 40, and 28 images, respectively. This makes them highly challenging test sets for source-free domain adaptation compared to CT and X-ray datasets. While our current focus is on demonstrating our method's effectiveness under these highly challenging conditions; Including X-rays and CT images represents a promising direction for future work to gain more insights.

***3) Did not demonstrate a significant enhancement in 2D results:***

We respectfully disagree with the comments regarding the lack of significant enhancements in results for 2D domain shifts. Our TT-SFUDA model demonstrates substantial improvements over Direct Testing, with increases in Dice scores of +5.5 (CHASE→HRF), +37.1 (CHASE→RITE), +7.0 (HRF→CHASE), and +17.3 (HRF→RITE). Additionally, when compared to traditional UDA methods, our approach exhibits superior performance. Notably, it outperforms the leading UDA method, BEAL, in three out of the four domain shifts, despite not utilizing source data during adaptation.
***1) More details on the datasets:***

We thank the reviewer for the comment and for better clarity, we have now included tables that gives details about the modality, demographic details, as well as the number of data in the appendix. The tables can also be seen here:

| 2D Datasets |   Ages   | Resolution | #Data |
|:-----------:|:--------:|:----------:|:-----:|
|    CHASE    |  5 to 16 |   999x960  |   28  |
|     RITE    | 25 to 90 |   768x584  |   40  |
|     HRF     | 21 to 90 |  3504x2336 |   18  |

| 3D Datasets | Modality | #Data |
|:-----------:|:--------:|:-----:|
|   BraTS-T1  |    T1    |  335  |
|  BraTS-T2   |    T2    |  335  |
| BraTS-FLAIR |   FLAIR  |  335  |
|  BraTS-T1ce |   T1ce   |  335  |

***2) Reason for not using validation set:***

We understand that having a validation set for hyper paramter tuning would increase the reliability. However, we would like to point out that the datasets that we work on are very small in number and splitting them further into train/val/test would lead to very less number of data for training as well as validation. For example, HRF dataset only has 18 images and doing a 70/15/15 split would result in only having 2 images for validation which might make the experiments very dependent on those 2 images chosen for validation.

***3) Reasons for why our method might not be optimial for certain shifts:***

While our method peforms better overall for most domain shifts, we do agree that for some cases our performance is not the best. However, we would like to point out that our performance is still comparable to the best performance for FLAIR->T2 domain shift. We believe this observation is because of the nature of certain domain shifts being difficlut for our proposed method to address. We also found that certain shifts like FLAIR -> T1 are very difficult and if the source training performance is itself very low, it becomes very difficult for any domain adaptation method to achieve a notable gain in performance.