Paper: PROGRAM: PROtotype GRAph Model based Pseudo-Label Learning for Test-Time Adaptation (ICLR-2024)

Reviewer comment (Evaluation):
The improvement of PROGRAM on the pseudo-labels is not quantitively evaluated. One key intuition behind PROGRAM is to improve pseudo-labels with PGM and RST. Though the ablation studies validate the necessity of PGM and RST in improving prediction accuracy, it is more convincing to explicitly evaluate the impact of these components on the quality of pseudo-labels.

Author response:
**Q1: The improvement of PROGRAM on the pseudo-labels is not quantitively evaluated.**

We appreciate the reviewer's important question regarding the quality of pseudo-labels generated by PGM. We conducted an additional experiment to explicitly evaluate the accuracy of the PGM module. The results are shown below.

Exp ID| Method            | VLCS      | PACS      | OfficeHome | TerraIncognita | Avg.   |
|---|-------------------|-----------|-----------|------------|-----------------|--------|
| #1 | ERM (ResNet-50)     |  76.71±0.50 | 83.21±1.14 | 67.13±0.99 | 45.93±1.34      | 68.25  |
| #2 | ERM + PGM (before fine-tune)          |  77.36±0.61| 84.16±0.93 | 67.69±1.35 | 46.38±1.79      | 68.90  |
| #3 | ERM + PROGRAM          |  78.17±0.92 | 90.78±0.40 | 69.05±0.66 | 47.84±1.23 | 71.46|
| #4 | ERM + PGM (after fine-tune)         |  78.73±1.27 | 91.30±0.63 | 69.57±0.88 | 48.31±1.42 | 71.98|


Previous TTA methods commonly utilize the pseudo-labels generated by ERM directly. By comparing results #1 and #2, we demonstrate that PGM can produce superior pseudo-labels compared to the original ERM (68.90 vs. 68.25). We then leverage these generated pseudo-labels for robust self-training, resulting in the improved performance of model #3 (71.46). In #4, we report the results obtained by applying PGM to generate the next round of pseudo-labels after fine-tuning. Consistent improvement is observed (71.98 vs. 71.46), highlighting the potential for iterative refinement. However, to ensure fair comparisons with other TTA methods, we do not report the results of #4 or the outcome of iterative refinement. These findings demonstrate that our method can produce more reliable pseudo-labels, leading to enhanced performance when compared with other TTA methods. We have included these results in the revised version.

Reviewer comment (Theory):
The similarity $w_{ij}$ in constructing the prototype graph is not well explained. In Eq. (2), the similarity $w_{ij}$ is defined over $p(v_i | v_j)$. In this case, $p(v_i | v_j) = p(v_j | v_i)$ for guaranteeing the symmetric similarity matrix. Equivalently, it might hold that $p(v_i , v_j) = p(v_i | v_j) p(v_j) = p(v_j | v_i) p(v_i)$ and then $p(v_i) = p(v_j)$ for all nodes. It is unclear whether it assumes that all samples follow a uniform distribution. This might be a strong assumption in real scenarios.

Author response:
**Q2: It is unclear whether it assumes that all samples follow a uniform distribution. This might be a strong assumption in real scenarios.**

We appreciate the reviewer's concern regarding the assumption of a uniform distribution. Indeed, this is the general assumption of TTA settings. TTA generally assumes that all samples in the test data come from the unique target domain and follow the same distribution. While this assumption may appear strong, it aligns with real-world applications. Transfering a source domain model to **one** target domain is very common, particularly for TTA. For example, the model might be pre-trained under controlled lab environments, but deployed to a certain camera installed at a certain position. As the camera view is fixed, the test data in this case can be assumed to come from the same domain. TTA is essential to enhance the pre-trained model's performance in **that** target domain and data distribution.

Reviewer comment (Evaluation):
Another concern is the computational efficiency of PROGRAM. It requires the graph construction and the matrix inverse computation. Besides, it also updates the "whole" feature extractors for RST. Both strategies might significantly increase the running time of PROGRAM for test-time adaptation, compared to other partial updating methods. Thus the trade-off between the effectiveness and efficiency of PROGRAM can be further analyzed, e.g., sparse graph construction, partial parameter updates, etc.

Author response:
**Q3: Concerns about computational efficiency. The trade-off between the effectiveness and efficiency of PROGRAM can be further analyzed, e.g., sparse graph construction, partial parameter updates, etc**

Thank you for your valuable suggestions. We have conducted additional experiments to further analyze the computational efficiency of PROGRAM and explore the trade-off between effectiveness and efficiency.

Firstly, we provide a breakdown of PROGRAM’s runtime, as suggested by Reviewer C9fo and DVoa. Our PGM module exhibits a relatively short computational time (comparing #1 and #5), making it efficient for reliable pseudo-label generation. And the main time consumption occurs during the back-propagation phase in RST. However, the computational time required for the RST module is comparable to other self-training modules that employ different loss functions (compare #7 with #8 and #9). Overall, our method demonstrates smaller computational costs compared to TTA methods such as SHOT (#2), SHOTIM (#3), and TSD (#4) that involve adjusting the whole model parameters. This highlights the practicality and feasibility of incorporating our method into real-world applications, as it achieves significant performance gains while maintaining reasonable computational efficiency.

**Absolute runtime performance analysis with ResNet-18 backbone on a single Titan XP GPU on the PACS dataset.**
| Exp ID | Method                           | Runtime (s)           |
| ------ | -------------------------------- | --------------------- |
| Baseline  |                             |                       |
| #1     | ERM (ResNet-18)                  | 11.15                 |
| #2     | ERM + SHOT (Liang et al, 2020)   | 11.15 + 20.97 = 32.12 |
| #3     | ERM + SHOTIM (Liang et al, 2020) | 11.15 + 20.73 = 31.88 |
| #4     | ERM + TSD (Wang et al, 2023)     | 11.15 + 19.73 = 30.88 |
| Ours   |                                  |                       |
| #5     | ERM + PGM                        | 11.15 + 1.21 = 12.36  |
| #6     | ERM + RST                        | 11.15 + 16.78 = 27.93 |
| #7     | **ERM + PGM + RST (PROGRAM)**   | 11.15 + 17.92 = 29.07 |
| #8     | ERM + PGM + CE Loss              | 11.15 + 17.54 = 28.69 |
| #9     | ERM + PGM + KL Loss              | 11.15 + 17.65 = 28.80 |




Additionally, following your suggestions, we have conducted additional experiments to balance accuracy and efficiency. The results are shown in the table below. 

1. We introduce a sparse graph construction method (#2). As PGM itself is relatively lightweight, the modifications to PGM have minimal impact on time consumption (28.62s vs 29.07s). 
2. We also investigate serveral partial parameter update approaches in the experiments. Specifically, we compared approaches that only fine-tune the linear classification layer (#3), only fine-tune the backbone (#4), and only fine-tune the Batch Normalization (BN) layer (#5). Based on the experimental results, fine-tuning only the linear layer achieves a good trade-off between accuracy (66.60%) and efficiency (13.05s), which could serve as an alternative according to different practical requirements. Compared with our original PROGRAM, it performs suboptimally on datasets with large domain gaps, such as PACS. Please refer to the response to Q3 of Reviewer DVoa for a more detailed explanation. Fine-tuning all the parameters is our default setting in order to achieve a model that performs well across all the scenarios.

|Exp ID| Method            |  VLCS      | PACS      | OfficeHome | TerraIncognita | Avg.   |Runtime (s)|
|---|-------------------|-----------|----------|------------|-----------------|--------|----|
|#1| ERM (ResNet-18)               |  74.88±0.46 |79.29±0.77| 62.10±0.31| 40.62±1.19| 64.22  |11.15|
|#2| +PROGRAM (sparse graph) |77.29±1.02  |87.74±1.45 |64.12±1.17  | 42.97±0.83 |68.03 |28.62|
|#3| +PROGRAM (fine-tune linear)         | 77.52±0.86 | 82.27±1.09 | 63.88±1.00 | 42.73±1.75    | 66.60   |13.05|
|#4| +PROGRAM (fine-tune backbone)         | 77.22±1.49 | 87.65±1.87 | 64.08±1.98 | 42.83±1.50   | 67.94  |28.99|
|#5| +PROGRAM (fine-tune BN layer)         | 76.73±1.20 | 87.04±1.63 | 63.21±0.97 |  42.16±1.23 |67.28  |28.86 |
|#6| +PROGRAM (our final version: fine-tune all)         | 77.75±1.37| 88.03±0.74| 64.59±0.85| 43.16±1.12| 68.38|29.07|

Reviewer comment (Experiments):
The Robust Self-Training (RST) combines both pseudo-labels and consistency regularization. The effectiveness of RST can be validated with more ablation studies. For example, compared to simple pseudo-labeling or consistency regularization, how can their combination perform better on the TTA benchmarks?

Author response:
**Q4: The effectiveness of RST can be validated with more ablation studies.**

Thank you for your valuable feedback. We have addressed your concerns by providing a theoretical analysis and conducting additional ablation studies to validate the effectiveness of our RST module.

1. **Theoretical Analysis**: We have included a detailed theoretical analysis in the Appendix (Sec. A3) to explain why our loss function is more effective. The analysis highlights how our RST module combines the advantages of two loss functions. When the generated pseudo-labels are reliable, we use the Cross-Entropy (CE) loss function, which treats the class with the maximum probability as the positive sample. This enables the self-training to converge quickly. When pseudo-labels are less reliable, we employ the Symmetric Cross Entropy (SCE) loss function. This analysis indicates that the SCE loss not only imposes consistency regularization but also simultaneously learns multiple categories with high predicted probabilities, thereby alleviating the impact of noisy pseudo-labels.
2. **Ablation Studies**: We have conducted additional ablation studies to validate the effectiveness of our RST module. We compare our RST approach with different strategies used in various TTA approaches, such as Tent, SHOT, PL, and T3A. The results are shown below. All experiments are conducted based on the ResNet-50 backbone. #1 is the baseline ERM. #2 replaces our RST module with the CE loss function, following Tent and SHOT. #3 improves upon #2 by introducing a confidence threshold=0.9, following PL. #4 modifies #2 by selecting the pseudo-labels with lower Shannon entropy, filtering out 20% of low-quality data, following T3A. #5 and #6 respectively apply KL loss and SCE loss for consistency regularization only. Comparing all these alternatives, our RST module (#7) achieves the best results.


ExpID| Method         | Used By   |  VLCS      | PACS      | OfficeHome | TerraIncognita | Avg.   |
|---|---------------|----|-----------|-----------|------------|-----------------|--------|
| #1 | ERM (ResNet-50)   | -   |  76.71±0.50| 83.21±1.14| 67.13±0.99| 45.93±1.34| 68.25|
| #2 | +CE Loss | Tent (Wang et al., 2021a) and SHOT (Liang et al., 2020) | 77.72±0.83 |90.45±0.37| 68.23±0.78 |46.92±1.04| 70.83|
| #3 | +CE Loss with confidence threshold=0.9| PL (Lee et al., 2013) | 77.76±1.26 |90.56±0.76| 68.28±0.89 |47.01±0.92| 70.90|
| #4 | +CE Loss with M-th largest entropy | T3A(Iwasawa & Matsuo, 2021)|77.77±1.71 |90.60±1.13| 68.32±0.94 |47.03±1.28| 70.93|
| #5 | +KL Loss | - |  77.61±1.52| 89.69±0.95| 68.96±1.54| 47.67±1.28| 70.98|
| #6 | +SCE Loss| - | 77.67±1.63| 89.72±1.01| 69.00±1.63| 47.80±0.77| 71.05|
| #7 | **+RST (PROGRAM)**| Ours | 78.17±0.92| 90.78±0.40| 69.05±0.66| 47.84±1.23| 71.46|

By providing a theoretical analysis and conducting additional ablation studies, we have thoroughly validated the effectiveness of our RST module.

Reviewer comment (Experiments):
The soft labels in Eq. (1) are estimated with the normalized model weights $c_k$ (prototypes). Compared to vanilla soft labels with the raw source model (with unnormalized weights), can these prototypes help provide better pseudo-labels?

Author response:
**Q5: Compared to vanilla soft labels with the raw source model (with unnormalized weights), can these prototypes help provide better pseudo-labels?**

It is important to note that the magnitudes of prototypes may vary. Without normalization, prototypes with larger magnitudes will exert a stronger influence on the computed value, which is not desired. We have conducted additional experiments to compare the results with (#3) and without (#2) normalization of prototypes. The experiments clearly demonstrate the necessity of using normalized prototypes instead of unnormalized ones.


|Exp ID| Method            | VLCS      | PACS      | OfficeHome | TerraIncognita | Avg.   |
|-----------------|-------------------|-----------|-----------|------------|-----------------|--------|
|#1| ERM (ResNet-50)  |  76.71±0.50 | 83.21±1.14 | 67.13±0.99 | 45.93±1.34      | 68.25  |
|#2| +PGM (with unnormalized weights) + RST   |  77.85±1.26 | 90.22±1.47 | 68.92±0.91 | 47.76±1.60      |  71.19 |
|#3| +PGM (with normalized weightst) + RST   |  78.17±0.92| 90.78±0.40| 69.05±0.66| 47.84±1.23| 71.46|

Reviewer comment (Theory):
Is the matrix $\mathbb{I} - \lambda \mathbf{Z} \mathbf{D}^{-1} \mathbf{Z}^T$ full rank to directly compute the inverse of this matrix?

Author response:
**Q6: Is the matrix $\bf{I} - \lambda ZD^{-1}Z^T$ full rank to directly compute the inverse of this matrix?**

With some calculations (please refer to Sec. A2 for more details), we can get:

$ |\mathbb{I} - \lambda \mathbf{Z}\mathbf{D}^{-1}\mathbf{Z}^T| = |\mathbf{B}_{K \times K}| \cdot |\mathbb{I} - \lambda (\mathbf{S} \mathbf{X} \mathbf{S}^T)| $

where $ \mathbf{B} \in \mathbb{R}^{N\times N} $ is a diagonal matrix with all values greater than zero, ensuring $|\mathbf{B}_{K \times K}| > 0 $. Additionally, $ \mathbf{X} \in \mathbb{R}^{N\times N} $ is also a diagonal matrix with all values greater than zero. Considering that $S_i$ and $S_j$ represent class probability vectors of two test samples and generally exhibit no linear correlation, we expect the rank of $ \mathbb{I}-\lambda (\mathbf{S} \mathbf{X} \mathbf{ S}^T) $ to be $N$. Consequently, $ |\mathbb{I} - \lambda \mathbf{Z} \mathbf{D}^{-1} \mathbf{Z}^T| \neq 0 $, indicating the invertibility of $ \mathbb{I} - \lambda \mathbf{Z} \mathbf{D}^{-1} \mathbf{Z}^T $.

Notably, in our experiments, we have not encountered non-invertible cases for this matrix. However, if a non-invertible situation arises, a small perturbation matrix $\mathbf{\Sigma} $ can be introduced to ensure invertibility: $(\mathbf{I} - \lambda \mathbf{Z} \mathbf{D}^{-1} \mathbf{Z}^T) + \epsilon \mathbf{\Sigma} $, where $ \epsilon $ is a positive number tending to zero. This regularization technique is commonly used, with $ \epsilon $ referred to as the regularization parameter.

Reviewer comment (Novelty):
The novelty of the paper remains to be discussed. Authors just use Prototype Graph Model directly and the strategy that selects some samples with CE loss and others with consistency regularization is commonly used in unsupervised learning. A similar paper [1] which also provides the derivation of Prototype Graph Model for few shot learning should be cited.

Author response:
**Q1-1: A similar paper (Zhu et al. 2023) [a] should be cited.**

Thanks for bring this paper (Zhu et al. 2023) to our attention. We appreciate your suggestion and include a citation to this paper in our revision. However, we would like to highlight the distinguishing aspects between our work and Zhu et al.:

1. Different research topics and settings: Zhu et al. focus on transductive few-shot learning, where LABELED support samples and ALL unlabeled queries are available, assuming training and testing occur in the SAME domain. Our paper, on the other hand, addresses online test-time adaptation, which involves fine-tuning a pre-trained model using a BATCH of unlabeled test data from the TARGET domain, without accessing to ANY labeled data.
2. Distinct graph construction: In Zhu et al. (2023), prototypes are not part of the graph and are updated separately using gradient descent. In contrast, we treat prototypes as graph vertices in our approach.
3. Varied prototype definitions: Zhu et al. (2023) initializes prototypes as mean vectors of labeled support samples since labeled data is available. In contrast, we initialize prototypes using the model weights of the classification layer.
4. Offline vs. online approach: Zhu et al. (2023) requires access to all unlabeled queries and performs iterative prototype updates in an offline manner. In contrast, in our online test-time adaptation, the unlabeled test set arrives in an online manner. Our re-initialization design ensures that our prototypes stay up to date with our model and maintain their representation of global class characteristics.

By highlighting these differences, we emphasize the unique contributions and focus of our work compared to Zhu et al. (2023).

[a] Hao Zhu, and Piotr Koniusz. Transductive Few-shot Learning with Prototype-based Label Propagation by Iterative Graph Refinement. In Proc. CVPR, pages 23996--24006, 2023. 

**Q1-2: Novelty concerns.**

The major contribution of this paper is to introduce a novel PROGRAM method specifically designed for test-time adaptation (TTA). PROGRAM addresses the issues of noisy pseudo-labels in the context of TTA by introducing the Prototype Graph Model (PGM) for pseudo-label generation and Robust Self-training (RST) for test-time fine-tuning. 

(1) Existing methods overlook either the global class prototype information, or the local neighboring features of test samples, leading to noisy pseudo-labels. To address this issue, we explore a novel scheme of using flexible graph representation to blend the benefits of both prototype-based and nearest-neighbor based pseudo-labeling. 

(2) We creatively combine pseudo-labeling (PL) and consistency regularization (CR) for self-training. We analyze the limitations of PL and CR separately. Based on the analysis, we propose RST to combine the benefits of hard pseudo-labels for accelerating model convergence and soft pseudo-labels for noise resistance.

Reviewer comment (Evaluation):
What is the accuracy and percentage of pseudo label after filtering by PGM?

Author response:
**Q3: What is the accuracy and percentage of pseudo label after filtering by PGM?**

We appreciate the reviewer's important question regarding the quality of pseudo-labels generated by PGM. First of all, We would like to clarify that our method does not filter out pseudo-labels. Instead, we introduce the integration of pseudo-labeling and consistency regularization to alternately fine-tuning the model in the RST module, which is one of the main innovations of our approach. By doing so, our method fully utilizes all pseudo-labels without explicitly filtering them. More details about this aspect can be found in the Introduction and Section 3.4 of our paper.

In response to the question, we also conducted an additional experiment to explicitly evaluate the accuracy of the PGM module. The results are shown below.

Exp ID| Method            | VLCS      | PACS      | OfficeHome | TerraIncognita | Avg.   |
|---|-------------------|-----------|-----------|------------|-----------------|--------|
| #1 | ERM (ResNet-50)     |  76.71±0.50 | 83.21±1.14 | 67.13±0.99 | 45.93±1.34      | 68.25  |
| #2 | ERM + PGM (before fine-tune)           |  77.36±0.61| 84.16±0.93 | 67.69±1.35 | 46.38±1.79      | 68.90  |
| #3 | ERM + PROGRAM          |  78.17±0.92 | 90.78±0.40 | 69.05±0.66 | 47.84±1.23 | 71.46|
| #4 | ERM + PGM (after fine-tune)          |  78.73±1.27 | 91.30±0.63 | 69.57±0.88 | 48.31±1.42 | 71.98|

Previous TTA methods commonly utilize the pseudo-labels generated by ERM directly (#1). The accuracy of the generated pseudo-labels by PGM is reported in #2. By comparing results #1 and #2, we demonstrate that PGM can produce superior pseudo-labels compared to the original ERM (68.90 vs. 68.25). We then leverage these generated pseudo-labels for robust self-training, resulting in the improved performance of model #3 (71.46). In #4, we report the results obtained by applying PGM to generate the next round of pseudo-labels after fine-tuning. Consistent improvement is observed (71.98 vs. 71.46), highlighting the potential for iterative refinement. However, to ensure fair comparisons with other TTA methods, we do not report the results of #4 or the outcome of iterative refinement. These findings demonstrate that our method can produce more reliable pseudo-labels, leading to enhanced performance when compared with other TTA methods. We have included these results in the revised version.

Reviewer comment (Writing):
Typo error: the caption of Table 4 : ↑ means higher is better.

Author response:
**Q4: Typo error.**

Thank you very much for pointing it out. We have fixed it in the revised version.

Reviewer comment (Experiments):
PGM lacks comparisons with non-parametric attention modules. Given the predefined graph G, the non-parametric attention module could first update each node in the graph with the help of the adjacency matrix, and then use the updated class prototypes to make predictions of each updated test feature.

Author response:
**Q2: PGM lacks comparisons with non-parametric attention modules.**

Thanks for the insightful suggestion. We have conducted additional experiments to compare PGM with non-parametric attention modules as suggested. The results are presented in the table below. The attention module have difficulty in efficiently exchanging message between prototypes represented by linear layers and the test features that may reside in different feature spaces and not follow the same distribution. So using the features from the test samples to update prototypes is not a good practice. By constructing graphs using probability distributions, we can avoid updating prototypes with test features. This method leads to better performance. These findings demonstrate that our PGM outperforms the attention module alternative (71.46 vs 70.73), confirming the effectiveness of our proposed prototype-based graph construction. 

|Exp ID| Method            | VLCS      | PACS      | OfficeHome | TerraIncognita | Avg.   |
|-------|-------------------|-----------|-----------|------------|-----------------|--------|
|#1| ERM (ResNet-50)     |  76.71±0.50 | 83.21±1.14 | 67.13±0.99 | 45.93±1.34      | 68.25  |
|#2| ERM + attention module +RST   |  77.68±1.51 | 89.85±1.22 | 68.29±1.35 | 47.12±2.03      |  70.73 |
|#3| ERM + PROGRAM          |  **78.17±0.92** | **90.78±0.40** | **69.05±0.66** | **47.84±1.23** | **71.46**|

Reviewer comment (Evaluation):
Could the author provide more insights into why PGM shows more reliable pseudo-labels? Especially, what does “more reliable” mean?

Author response:
**Q3: Could the author provide more insights into why PGM shows more reliable pseudo-labels? Especially, what does “more reliable” mean?**

Thanks for the reviewer's question. The term "more reliable" refers to higher accuracy in this context. Please kindly note that the motivation behind the PGM module has been extensively discussed in our paper. We have provided a thorough analysis of both prototype-based pseudo-labeling and nearest-neighbor pseudo-labeling, highlighting their respective strengths and limitations. PGM combines the advantages of both approaches by incorporating global representative features of prototypes and local information from neighboring test data through a flexible graph representation.

To further demonstrate the effectiveness of PGM in generating more reliable pseudo-labels, we conduct an additional experiment. The quality of pseudo-labels generated by PGM was explicitly evaluated in the following results.


Exp ID| Method            | VLCS      | PACS      | OfficeHome | TerraIncognita | Avg.   |
|---|-------------------|-----------|-----------|------------|-----------------|--------|
| #1 | ERM (ResNet-50)     |  76.71±0.50 | 83.21±1.14 | 67.13±0.99 | 45.93±1.34      | 68.25  |
| #2 | ERM + PGM (before fine-tune)           |  77.36±0.61| 84.16±0.93 | 67.69±1.35 | 46.38±1.79      | 68.90  |
| #3 | ERM + PROGRAM          |  78.17±0.92 | 90.78±0.40 | 69.05±0.66 | 47.84±1.23 | 71.46|
| #4 | ERM + PGM (after fine-tune)          |  78.73±1.27 | 91.30±0.63 | 69.57±0.88 | 48.31±1.42 | 71.98|

Previous TTA methods commonly utilize the pseudo-labels generated by ERM directly (#1). The accuracy of the generated pseudo-labels by PGM is reported in #2. By comparing results #1 and #2, we demonstrate that PGM can produce superior pseudo-labels compared to the original ERM (68.90 vs. 68.25). We then leverage these generated pseudo-labels for robust self-training, resulting in the improved performance of model #3 (71.46). In #4, we report the results obtained by applying PGM to generate the next round of pseudo-labels after fine-tuning. Consistent improvement is observed (71.98 vs. 71.46), highlighting the potential for iterative refinement. However, to ensure fair comparisons with other TTA methods, we do not report the results of #4 or the outcome of iterative refinement. These findings demonstrate that our method can produce more reliable pseudo-labels, leading to enhanced performance when compared with other TTA methods. We have also included these results in our revised version for further clarification.

Reviewer comment (Presentation):
The explanation for Figure 1 appears to lack details. A clearer description would be helpful, such as specifying that the red line represents the decision boundary. Moreover, the meaning of the red dashed line in 1(b) is ambiguous.

Author response:
**Q1: Suggestions on improving the clarity of Figure 1.**

Thank you for your valuable feedback. We have made the necessary refinements to Figure 1 to enhance its clarity and improve readability.

Reviewer comment (Presentation):
The meaning of the symbols ‘+’ used in Tables 1 and 2 is not clear. It would be helpful if their meaning can be clarified.

Author response:
**Q2: The meaning of the symbols ‘+’ used in Tables 1 and 2 is not clear.**

We apologize for any confusion caused. The symbol '+' indicates the application of the TTA method to the ERM baseline method. Each row in Tables 1 and 2 represents the evaluation accuracy of the ERM baseline combined with the respective TTA method.

Reviewer comment (Evaluation):
While PROGRAM's runtime is competitive as a whole feature extractor, it noticeably lags behind some partial ones like T3A. Given that the performance edge of PROGRAM over T3A isn't substantial in some tasks (as shown in Table 1), could you shed light on the specific scenarios where PROGRAM would be the preferred choice? Essentially, under what circumstances should one opt for PROGRAM, even at the expense of computational speed?

Author response:
**Q3: Could you shed light on the specific scenarios where PROGRAM would be the preferred choice compared with partial-update methods? Under what circumstances should one opt for PROGRAM, even at the expense of computational speed?**

Our method, PROGRAM, excels in scenarios where there is a substantial domain shift between the source and target data. Partial-update methods, such as T3A and TAST, assume that the feature extractor backbone trained on the source domain performs well on the target domain, and only adjust the final linear classification layer. However, when the domain shift is significant, the assumption does not hold and may yield poor results without adapting the backbone.

Our experimental results support this analysis. Li et al. [a] pointed out that the VLCS dataset has a relatively smaller domain shift compared to the PACS dataset. In Table 1 of our main text, we observe that PROGRAM demonstrates a more pronounced improvement on datasets with significant domain shifts, such as the PACS dataset (+7.2% compared to T3A), compared to datasets with smaller domain shifts like VLCS (+0.49% compared to T3A). Moreover, on image corruption benchmarks with substantial domain gaps, our approach significantly outperforms T3A (average improvement of 20.34%). While the improvement on datasets with smaller domain gaps may not be as evident, there is still a notable and consistent enhancement.

As suggested by Reviewer XKXi, we also investigated a partial-update variant of PROGRAM, which only fine-tuned the linear classification layer (#5). Based on the experimental results below, fine-tuning only the linear layer (#5) achieves a good trade-off between accuracy (66.60%) and efficiency (13.05s). For instance, it achieves better accuracy than TAST (66.60% vs 66.39%), while being more efficient (13.05s vs 18.07s). 


| Exp ID | Method                                       | VLCS       | PACS       | OfficeHome | TerraIncognita | Avg.  | Runtime (s) |
| ------ | -------------------------------------------- | ---------- | ---------- | ---------- | -------------- | ----- | ----------- |
| #1     | ERM (ResNet-18)                              | 74.88±0.46| 79.29±0.77| 62.10±0.31 |40.62±1.19| 64.22|11.15|
| #2     | +T3A (Iwasawa & Matsuc, 2021)   | 77.26±1.49| 80.83±0.67 |63.21±0.50| 40.20±0.60| 65.38|11.73|
| #3     | +TAST (Jang et al. 2023)       | 77.27±0.67 |81.94±0.44 |63.70±0.52| 42.64±0.72|66.39|18.07|
| #4     | +TAST-BN (Jang et al. 2023)       |  75.21±2.36| 87.07±0.53| 62.79±0.41| 39.43±2.24| 66.13 |85.08|
| #5     | +PROGRAM (fine-tune linear)                 | 77.52±0.86 | 82.27±1.09 | 63.88±1.00 | 42.73±1.75    | 66.60 | 13.05       |
| #6     | +PROGRAM (our final version: fine-tune all) | 77.75±1.37| 88.03±0.74| 64.59±0.85 |43.16±1.12| 68.38|29.07|



[a] Li, Da, et al. "Deeper, broader and artier domain generalization." Proceedings of the IEEE international conference on computer vision. 2017.

Reviewer comment (Evaluation):
Could you provide a breakdown of PROGRAM's runtime between PGM and RST (Table 6)?

Author response:
**Q4: Could you provide a breakdown of PROGRAM's runtime?**

Thanks for the advice. We report the breakdown of PROGRAM's runtime below.

**Absolute runtime performance analysis with ResNet-18 backbone on a single Titan XP GPU on the PACS dataset.**
| Exp ID | Method                           | Runtime (s)           |
| ------ | -------------------------------- | --------------------- |
| Baseline  |                             |                       |
| #1     | ERM (ResNet-18)                  | 11.15                 |
| #2     | ERM + SHOT (Liang et al, 2020)   | 11.15 + 20.97 = 32.12 |
| #3     | ERM + SHOTIM (Liang et al, 2020) | 11.15 + 20.73 = 31.88 |
| #4     | ERM + TSD (Wang et al, 2023)     | 11.15 + 19.73 = 30.88 |
| Ours   |                                  |                       |
| #5     | ERM + PGM                        | 11.15 + 1.21 = 12.36  |
| #6     | ERM + RST                        | 11.15 + 16.78 = 27.93 |
| #7     | **ERM + PGM + RST (PROGRAM)**   | 11.15 + 17.92 = 29.07 |
| #8     | ERM + PGM + CE Loss              | 11.15 + 17.54 = 28.69 |
| #9     | ERM + PGM + KL Loss              | 11.15 + 17.65 = 28.80 |


Our PGM module exhibits a relatively short computational time (comparing #1 and #5), making it efficient for reliable pseudo-label generation. And the main time consumption occurs during the back-propagation phase in RST. However, the computational time required for the RST module is comparable to other self-training modules that employ different loss functions (compare #7 with #8 and #9). Overall, our method demonstrates smaller computational costs compared to TTA methods such as SHOT (#2), SHOTIM (#3), and TSD (#4) that involve adjusting the whole model parameters. This highlights the practicality and feasibility of incorporating our method into real-world applications, as it achieves significant performance gains while maintaining reasonable computational efficiency.

Reviewer comment (Evaluation):
From Table 3, RST seems to have a small impact on improving results. Furthermore, ResNet-50 on its own shows good results. How do you justify the improvements brought by PGM and RST given their relatively modest contribution to performance?

Author response:
**Q5-1: From Table 3, RST seems to have a small impact on improving results. How do you justify the improvements brought by PGM and RST given their relatively modest contribution to performance?**

Please kindly note that in Table 1, with ResNet-50 as the backbone, TSD (Wang et al., 2023) improves upon TAST (Jang et al., 2023) by 0.43%, and TAST improves upon T3A (Iwasawa & Matsuc, 2021) by 0.69%. These improvements are considered substantial in the field of test-time adaptation (TTA). In comparison, RST alone brings an improvement of 0.63%. Moreover, when combined with PGM, our method (PROGRAM) achieves a more significant improvement over the previous state-of-the-art TSD by 1.57%. We believe that the performance boost achieved by PROGRAM is substantial and demonstrates its effectiveness in advancing the state-of-the-art in TTA.


**Q5-2: ResNet-50 already shows good results.**
While ResNet-50 may achieve good results in scenarios with small domain gaps, test-time adaptation (TTA) becomes crucial when dealing with large domain gaps. For instance, the PACS dataset exhibits a strong domain gap. Our approach shows a significant improvement over the ResNet-50 baseline (90.78% vs 83.21%) on PACS. Particularly, on image corruption datasets, our approach demonstrates substantial enhancement, with an average improvement of 23.91%. These results validate the necessity and effectiveness of our approach in addressing challenging scenarios with significant domain gaps.