Paper: Continual Test-Time Adaptation by Leveraging Source Prototypes and Exponential Moving Average Target Prototypes (ICLR-2024)

Reviewer comment (Novelty):
A significant concern regarding this paper is its lack of technical innovation. Despite being an application paper, the method merely applies EMA for continual test time adaptation, employing standard techniques for selecting reliable test exemplars, computing class prototypes, and calculating EMA penalties. These methodologies are well-established within the existing literature.

Author response:
Please refer to the official comment to all authors about the novelty of this work.

Reviewer comment (Evaluation):
The empirical evidence presented in the paper lacks persuasiveness. A substantial performance boost could have justified the paper's simplicity and application-oriented nature. However, the minor improvement over EATA and RMT, as indicated primarily in Table 1-2 (i.e., less than +0.5% accuracy), does not substantiate the approach's effectiveness convincingly.

Author response:
We respectfully disagree with the assertion that our proposed method yields marginal performance gains. In the case of ImageNet-C, our proposed term enhances the mean accuracy across all target domains by more than 1% for all three applied methods (EATA, CoTTA, and RMT).  More than 1% performance gain is considered significant, especially given the challenging nature of the continual test-time adaptation setting and the dataset containing 1000 classes. Additionally, for CIFAR100-C, our proposed term leads to meaningful performance improvements for both EATA and CoTTA. The only scenario where marginal performance gains are observed is with RMT on CIFAR100-C, attributed to its intricate incorporation of several loss terms.  
We are uncertain about the reason for stating that there is an accuracy gain of less than +0.5%.  

Moreover, we conducted through ablation study and analysis to validate the efficacy of our proposed method. We respectfully request the reviewer to check our ablation study and analysis as well.

Reviewer comment (Experiments):
One notable omission in the paper is the absence of a comparison or reference against a significant baseline, namely NOTE [1], which is a robust continual test-time adaptation method designed to handle temporal correlations. This baseline is highly relevant to the authors' objectives and should have been included for a comprehensive evaluation.

Author response:
NOTE addresses the temporal correlation of TTA, primarily focusing on the imbalanced class distribution of incoming target data. While this differs slightly from our problem, which specifically addresses the domain shift of target data, we plan to incorporate NOTE as one of the baseline methods in the revised version. Additionally, we will explore other datasets that closely resemble the natural shifts of continual test-time adaptation setting.

Reviewer comment (Reproducibility):
How are the hyper parameters tuned? Is the source data used to initialize $P^s$ employing training data or validation data?

Author response:
(1a) Hyper-parameters are tuned via grid search. Source data used to initialize $P^s$ are sub-samples of training data. It is stated in the section 3.2 of our original submission.

Reviewer comment (Novelty):
This paper needs to properly state its contributions over TTAC (the other clustering approach in the literature).

Author response:
(1b) TTAC assumes that each class cluster follows a Gaussian distribution. TTAC pre-calculate the mean and covariance ($\mu_s \in \mathbb{R}^d$ and $\Sigma_s \in \mathbb{R}^{d \times d}$, where $d$ is the dimension of the feature) for each class of source domain data, computing Gaussian distributions for each source class prior to test-time adaptation. At test-time, it uses the target data to compute the Gaussian distribution for each class cluster of target domain as well. It uses the distributions of source domain class clusters as anchors for the distributions of target domain class clusters to match against. The anchoring of cluster is achieved by directly minimizing the KL-Divergence between the source and the target distributions.  

Computing Gaussian distribution for each class cluster and minimizing KL-Divergence between the two distributions require computation overhead. We show that it indeed requires significant time to adapt a single batch in Figure 2 while our proposed method requires a small amount of time for adaptation. Pleas refer to the difference between the green and the blue plot of Figure 2.  
We also show that in the main performance comparison tables (Table 1 and Table 2), ours outperforms TTAC in both benchmarks, showing superiority over TTAC in both accuracy and adaptation time.  

Unlike TTAC, our proposed method does not require the computation of intricate Gaussian distributions or the KL-Divergence between source and target distributions. Instead, it directly minimizes the MSE distance between the target feature and the source prototypes ($P^s$), akin to $\mu_s$ in TTAC. This makes our method simpler, more efficient, and achieves superior performance over TTAC when combined with $\mathcal{L}_{ema}$.

Reviewer comment (Reproducibility):
It is unclear whether the predictions in Algorithm 1 line 5 are adjusted before the output phase (line Ensure) as the predictions $z$ are returned as $\hat y$. Further, the algorithm returns the set of predictions for all data-points and all domains rather than conducting the evaluation in an online manner (return the predictions batch by batch).

Author response:
(1c) Our algorithm returns the predictions batch by batch in an online manner. Line "Ensure" indicates that our algorithm ensures the prediction outputs for all the inputs from all the target domains. It does "not" mean that the predictions are returned all together at "once" when the test-time adaptation is finished.

Reviewer comment (Evaluation):
It is unclear why the proposed components degrade the performance of EATA under small batch-sizes.

Author response:
(2b) When employing small batch sizes, the model exhibits low performance in batch accuracy. For a majority of batches (iterations), the accuracy is either 0% or less than 50%. This low accuracy hampers the construction of EMA target prototypes with accurate pseudo-labels, as well as the source alignment loss. Furthermore, given the reduced number of reliable samples after entropy-based sample selection due to low accruacy, the target prototypes are infrequently updated and remain unchanged. These challenges result in diminished performance of our proposed loss terms when using small batch sizes.

Reviewer comment (Evaluation):
While EATA does not regularize for its features to be clustered (unlike the proposed approach), they are still very competitive (better in discriminating different clusters) when the proposed components are absent.

Author response:
(2c) EATA uses source samples to compute fisher importance of model weights before performing test-time adaptation. With the computed fisher importance, it employs anti-forgetting loss which prevents the parameters of the model from changing too much from their source trained weights during the test-time adaptation. It also selects reliable and non-redundant samples to facilitate efficient entropy minimization loss.

Reviewer comment (Experiments):
The analysis and comparison of the proposed method should be against EATA+TTAC rather than EATA (e.g. in Figures 4 and 5).

Author response:
(2d) The analysis in Figure 4 and 5 are conducted between EATA and EATA+Ours to show the efficacy of our proposed terms. We wanted to analyze how our proposed terms affect the model by ablating the proposed terms. If we compare between EATA+TTAC and EATA+Ours, we can not properly analyze how our proposed method has effect on the model.

Reviewer comment (Experiments):
Experiments with different and more powerful architectures that do not use batch normalization (e.g. ViT) layers are missing.

Author response:
(2e) We will try to include experiments using different network architecture in the future version.

Reviewer comment (Writing):
The problem definition is not clear. Both $k$ and $t$ are used to refer to time.

Author response:
(3a) $k$ refers to the domain index as mentioned in the paper while $t$ refers to the training iteration.

Reviewer comment (Presentation):
The introduction should state clearly the contributions this work provides.

Author response:
(3b) Our contributions are well stated in the last paragraph of the introduction.

Reviewer comment (Presentation):
The related work section is missing from the main paper and put in the appendix. It is essential to have this section to clearly position this work in the literature.

Author response:
(3e) We will include the related work in the main paper in future version.

Reviewer comment (Novelty):
While the simplicity of the proposed approach is appreciated, the performance gains are generally very marginal (with best choice of hyper parameters), questioning the usefulness of the proposed method.

Author response:
We respectfully disagree with the assertion that our proposed method yields marginal performance gains. In the case of ImageNet-C, our proposed term enhances the mean accuracy across all target domains by more than 1% for all three applied methods (EATA, CoTTA, and RMT).  More than 1% performance gain is considered significant, especially given the challenging nature of the continual test-time adaptation setting and the dataset containing 1000 classes. Additionally, for CIFAR100-C, our proposed term leads to meaningful performance improvements for both EATA and CoTTA. The only scenario where marginal performance gains are observed is with RMT on CIFAR100-C, attributed to its intricate incorporation of several loss terms.

Reviewer comment (Experiments):
Compared to CoTTA, where unsupervised test-time adaptation methods function without relying on source domain data, this paper's reliance on pre-computed source prototypes from the source domain data seems to be an uneven ground. It raises questions about the comparability and fairness of the presented method relative to others, like CoTTA, which function without such dependencies.

Author response:
**Regarding Comparability**

As we addressed in response to reviewer JC3N, it is crucial to highlight that baseline methods like TTAC, EATA, RMT all leverage source domain data according to their proposed methodologies. Nevertheless, we acknowledge the valid point raised by you, suggesting that comparing with methods that do not utilize any source domain data, such as CoTTA, may appear unfair.

In the table below, we show that even in the absence of source domain data, specifically without employing $\mathcal{L}_{src}$ 

and solely utilizing $\mathcal{L}_{ema}$, there is a performance improvement. 

This table shows the average accuracy over the 16 test domains of ImageNet-C benchmark.

|Method|Mean Acc. over the 16 test domains.|
|----|----|
|$\mathcal{L}_{ema}$|44.00%|
|$L_{ema} + \mathcal{L}_{src}$|45.96%|
|EATA|49.81%|
|EATA+$\mathcal{L}_{ema}$|50.62%|
|EATA+**Ours**|51.32%|
|CoTTA|37.42%|
|CoTTA+$\mathcal{L}_{ema}$|46.39%|
|CoTTA+**Ours**|46.91%|
|RMT|41.68%|
|RMT+$\mathcal{L}_{ema}$|42.29%|
|RMT+**Ours**|42.78%|

Notably, when $\mathcal{L}_{ema}$  is applied independently of the baseline method, it already outperforms CoTTA (44.00% vs. 37.42%).

Also, CoTTA+$\mathcal{L}_{ema}$ shows remarkable performance compared to CoTTA, 

supporting the validity of proposed $\mathcal{L}_{ema}$.

$\mathcal{L}_{src}$ can be additionally employed to further boost the performance when a sub-sample of source domain data are available.

It is essential to underscore that CoTTA necessitates additional computation and memory resources due to its teacher-student framework and augmentation strategy. 

This results in a longer adaptation time for a single batch, as illustrated in Figure 2 of our original submission.

Reviewer comment (Novelty):
The various loss components highlighted in the paper have previously been discussed in many other studies within the Domain Adaptation (DA) and Test-Time Adaptation (TTA) fields. This gives an impression that the proposed method might just be a combination of existing techniques, akin to piecing together different methods like A+B+C. Additionally, the usage of $L_{ema}$, which is commonly found in unsupervised DA, doesn't appear novel.

Author response:
**Concerning Novelty**

Please refer to the official comment to all authors about the novelty of this work.

Reviewer comment (Presentation):
Figure 1 in the paper comes across as overly intricate and disorganized. A reader would need to invest significant effort to discern and correlate the various methods depicted, which hampers the immediate understanding of the paper's methodology.

Author response:
**Clarity of Figure 1**

We will elaborate the Figure 1 for better understanding of the proposed methodology in the future version. 
We consider having a sperate figure for each proposed loss term and further describe each loss term in more detail.

Reviewer comment (Writing):
The $L_{unsup}$ seems to be inadequately defined in the paper. Even though the authors touched upon it in the "Problem definition" section, a more explicit formula or representation would have been beneficial for clarity and a more straightforward comprehension.

Author response:
**Formulation of $\mathcal{L}_{unsup}$**

As stated in the problem definition, $\mathcal{L}_{unsup}$ can take a form of entropy minimization loss, akin to the approach in TENT and EATA. Alternatively, it can adopt the form of a distillation loss from the teacher network to the student network, as employed in CoTTA.

Entropy minimization loss term is defined as follows:

$$
\mathcal{L}_{ent}=-\sum^C_c (\sigma(g(x^t))_c \cdot \log(\sigma(g(x^t))_c))
$$

Where $\sigma$ is the softmax operation and  $g_{\theta}(x^t)_c$  

refers to the $c$-th element of the produced logit $z^t=g_{\theta}(x^t) \in \mathbb{R}^C$. 

It is basically a loss to minimize the entropy of the produced logit.

Distillation loss term (or the consistency loss as named in CoTTA) is defined as follows:

$$
\mathcal{L}_{ent}=-\sum^C_c(\sigma(g_t(x^t))_c \cdot \log(\sigma(g_s(x^t))_c))
$$

Where $g_{t}$ and $g_{s}$ refer to the teacher and the student networks.

Reviewer comment (Novelty):
Concerning novelty, the various loss components presented in the paper seem to have been discussed in previous DA and TTA research. The authors should elaborate on what sets their method apart, particularly regarding its novelty, and how the incorporation of these loss components enhances the uniqueness and effectiveness of the proposed method.

Author response:
**Concerning Novelty**

Please refer to the official comment to all authors about the novelty of this work.

Reviewer comment (Reproducibility):
Computating the source prototypes requires the source domain data. So, the proposed approach will not work for any off-the-shelf pre-trained model without the source domain data.

Author response:
**The usage of source domain data**

First, we want to mention that our baseline methods also require source domain data. TTAC requires computing source cluster parameters ($\mu_s \in \mathbb{R}^d$ and $\Sigma_s \in \mathbb{R}^{d \times d}$, where $d$ is the dimension of the feature), the mean and covariance, with the source data prior to test-time adaptation. EATA requires source domain data to compute the importance weights for the anti-forgetting regularization before deploying the model to the target domains. RMT also employs source replay loss which accesses the source domain data during test-time adaption. 
CoTTA, TSD, and other methods do not necessitate source domain data, yet they exhibit comparatively lower performance. 

In the table below, we show that even in the absence of source domain data, specifically without employing $\mathcal{L}_{src}$

and solely utilizing $\mathcal{L}_{ema}$, there is a performance improvement. 

$\mathcal{L}_{src}$ can be further incorporated into training the model when there are some source samples available.

This table shows the average accuracy over the 16 test domains of ImageNet-C benchmark.

|Method|Mean Acc. over the 16 test domains.|
|----|----|
|$\mathcal{L}_{ema}$|44.00%|
|$L_{ema} + \mathcal{L}_{src}$|45.96%|
|EATA|49.81%|
|EATA+$\mathcal{L}_{ema}$|50.62%|
|EATA+**Ours**|51.32%|
|CoTTA|37.42%|
|CoTTA+$\mathcal{L}_{ema}$|46.39%|
|CoTTA+**Ours**|46.91%|
|RMT|41.68%|
|RMT+$\mathcal{L}_{ema}$|42.29%|
|RMT+**Ours**|42.78%|

Reviewer comment (Theory):
Utilizing class prototypes is limited to classification tasks and not generalizable to other tasks.

Author response:
Thanks for your response.

We think the notion of class is inevitable. Without the information of classes, we can not even define the output space of the classifier (the last linear layer). TTA assumes that the number of classes and their information are given and only deals with the domain shifts problem.

Reviewer comment (Experiments):
Utilizing class prototypes is limited to classification tasks and not generalizable to other tasks. Can this approach be generalizable to other tasks, such as segmentation?

Author response:
**Generalization to other tasks**

While certain specific implementations need further development, we believe that our proposed method holds the potential for generalization to other tasks, such as semantic segmentation. 
Given that segmentation involves pixel-wise predictions, the proposed loss terms $\mathcal{L}_{src}$ 

and $\mathcal{L}_{ema}$ can be applied in a pixel-wise manner.

We can construct the source prototypes and the EMA prototypes with the features given prior to the prediction head of the segmentation network. 
We can collect reliable pixel-wise features exploiting the prediction results from the segmentation head, then we can aggregate them to generate the proposed prototypes.  
Then, with the generated prototypes, we can compute the loss terms $\mathcal{L}_{src}$ 

and $\mathcal{L}_{ema}$ on predictions given by the segmentation network.

Reviewer comment (Novelty):
My major concern is about the novelty. Prototypical alignment has been extensively explored in previous domain adaptation methods. The major difference between the proposed approach and those prior efforts seems marginal. I suggest that the authors provide detailed comparisons and showcase why their proposed approach is preferable when applied to test-time adaptation scenarios.

Author response:
**The novelty of the work**

Please refer to the official comment to all authors about the novelty of this work.