Paper: Test Time Adaptation with Auxiliary Tasks (ICLR-2024)

Reviewer comment (Theory):
The arguments for availability of _unlabeled_ source data at inference, as opposed to storing some training data in a replay buffer, are very unconvincing.

Author response:
**Regarding the use of unlabeled data instead of storing labeled examples in a replay buffer**: We thank the reviewer for this remark. While we agree that in some applications, storing labeled training examples is feasible, in other scenarios (such as CLIP [A]), labeled training data can be a private property that cannot be accessed. This was the main rationale behind using unlabeled data for our DISTA. Nevertheless, and as per the reviewers request, we conducted experiments on DISTA with replacing the distillation loss with a cross entropy loss with the ground truth labels. The overall objective function for DISTA-Labeled is:

$\min_{\theta}  \underset{x_t\sim \mathcal S}{\mathbb E} \lambda_t(x_t) E\left(f_\theta(x_t)\right)  +   \underset{(x_s, y_s) \sim  \mathcal D_s}{\mathbb E} \lambda_s(x_s) CE\left(f_\theta(x_s),y_s\right)$

Our results on ImageNet-C benchmark are in the table below where we report error rates. 

|            	| Gauss | Shot  | Impul | Defoc | Glass | Motion| Zoom  | Snow  | Frost | Fog   | Bright| Contr | Elastic | Pixel | Jpeg  | Avg.  |
|----------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|---------|-------|-------|---------|
| EATA       	| 64.0  | 62.1  | 62.5  | 66.9  | 66.9  | 52.5  | 47.4  | 48.2  | 54.2  | 40.2  | 32.2  | 54.6  | 42.2	| 39.2  | 44.7  | 51.9	|
| DISTA + Labels | 62.7  | 60.9  | 60.9  | 66.0  | 66.1  | 50.7  | 46.9  | 47.4  | 53.6  | 39.2  | 31.9  | 54.9  | 41.5	| 38.6  | 44.2  | 51.0	|
| DISTA      	| 62.2  | 59.9  | 60.6  | 65.3  | 65.3  | 50.4  | 46.2  | 46.6  | 53.1  | 38.7  | 31.7  | 53.2  | 40.8	| 38.1  | 43.5  | 50.4	|

We observe that incorporating labeled data in DISTA instead of our distillation loss does not improve its results over using an unsupervised objective (while still providing state-of-the-art results compared to other baselines in our paper). We included this experiment in Appendix B.5 and referenced it in the main paper.

Reviewer comment (Evaluation):
The lookahead analysis tool only measures the correlation between entropy of the auxiliary task and test data, but doesn't inform us of the test accuracy.

Author response:
**Regarding the lookahead analysis**: We appreciate the reviewer’s comment on the lookahead analysis. While indeed, one should quantify the lookahead based on performance measures  (e.g. accuracy), the labels at test time are not provided by the stream. We emphasize that this is an inherent challenge in the test-time-adaptation problem where the model is required to predict and leverage *unlabeled* data for the adaptation strategy. Hence, and following the TTA setup, we extended the lookahead analysis to this unsupervised setup by defining it in terms of entropy as a proxy metric, that is known to be highly correlated to test accuracy.
We believe that providing the lookahead in terms of accuracy might make the analysis less practical for the TTA setup where labels are never available.

Reviewer comment (Experiments):
While prior works have shown the correlation between test data entropy and accuracy, this cannot be taken for granted, as the paper doesn't show when the proposed method _doesn't_ work, i.e., when having source distillation is not useful or is harmless.

Author response:
**Regarding the improvement that entropy minimization on source data can provide (Equation 2)**: We hypothesize that during the adaptation process, the adapted model overfits to the presented distribution from the stream while increasing its uncertainty on the clean data. However, including a simple entropy minimization on source data might mitigate partially this effect by providing a more stable adaptation process. We note that we confirmed the usefulness of this approach in Table 6 in the main paper where Aux-Tent improved the error rate over Tent by more than 1% on Gaussian, Shot, and Impulse noises, and by 0.6% on average on ImageNet-C confirming the earlier lookahead analysis in Figure 1 (a).

Reviewer comment (Experiments):
The ablation only shows the batch size more than 8. How does the method behave when the batch size is 1? MEMO shows this is possible with special handling of batchnorm parameters.

Author response:
**Regarding experiments with batch size smaller than 8**: We thank the reviewer for this comment. We note that we included experiments under batch size 1 in Table 9 in the appendix, where we compare DISTA against the state-of-the-art SAR under that setup. DISTA provided significant performance gains (around 5% reduction in error rate).

Reviewer comment (Reproducibility):
What are the trainable parameters? Is it only normalization or the whole network? What are the effects?

Author response:
**Regarding the trainable parameters**: We thank the reviewer for this comment. As mentioned in appendix B.1, DISTA only updates the learnable parameters of normalization layers. This choice is common practice in the TTA literature [A, B, C]. We agree this is an important detail and decided to move this experimental detail to the main paper for better clarity.

Reviewer comment (Experiments):
What is the importance of the $E_0$ parameter on the data filtration process? While this is proposed in prior works, a study of this in the current work could be illuminating.

Author response:
**Regarding the importance of $E_0$**: We appreciate the reviewer’s comment. E_0 plays a vital role in filtering unreliable samples (samples with high entropy). We conducted  experiments with two values of E_0 (0.4*log(1000), 0.5*log(1000)) and report the results in the table below. 

| E0   | Gauss     	| Shot      	| Impul     	| Defoc     	| Glass     	| Motion    	| Zoom      	| Snow      	| Frost     	| Fog       	| Bright    	| Contr     	| Elastic   	| Pixel     	| Jpeg      	| Avg.      	|
|----------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|
| 0.4 log(1000)   | 62.9      	| 60.7      	| 61.4      	| 65.8      	| 65.9      	| 51.2      	| 46.5      	| 47.6      	| 53.7      	| 39.3      	| 31.7      	| 54.3      	| 41.6      	| 38.5      	| 44.1      	| 51.0      	|
| 0.5 log(1000)	| 62.2      	| 59.9      	| 60.6      	| 65.3      	| 65.3      	| 50.4      	| 46.2      	| 46.6      	| 53.1      	| 38.7      	| 31.7      	| 53.2      	| 40.8      	| 38.1      	| 43.5	| 50.4


We observe that varying E_0 results in performance variation due to filtering more/less samples from the update step. This result is in Table 11 in the Appendix.

Reviewer comment (Theory):
Intuition behind eq, (3) is not straightforward. In particular, $\theta_{t+1}$ is obtained by updating ${\theta_t}^c$ using the gradients w.r.t  $\theta$. Discussing the intuition behind it would be helpful.

Author response:
**Regarding the intuition behind Equation (3)**: The main intuition behind Equation (3) is to solve the optimization problem in Equation (2) in an alternating optimization manner. This is because we want to analyze the impact of the auxiliary task (second term in Equation 2), and conducting this alternating optimization approach grant us an access to the intermediate model parameters $\theta_t^c$ and allows us to compare the performances of $\theta_{t+1}$ and $\theta_t^c$. We edited the paragraph before equation (3) to clarify that. Note that the expectations in equation (3) are taken over different data distributions (resp. corrupted and clean data distributions), and while the gradients w.r.t. $\theta$ are used to update $\theta^c_t$ to get $\theta_{t+1}$, the gradient is evaluated at $\theta_t^c$. That is, 

$\theta_t^c = \theta_t - \gamma \nabla_\theta [E(f_\theta(x_t)]|_{\theta = \theta_t}$

$\theta_{t+1} = \theta_t^c - \gamma \nabla_\theta [E(f_\theta(x_t)]|_{\theta = \theta_t^c}$

We hope that this has clarified the intuition behind Equation (3).

Reviewer comment (Experiments):
It is mentioned in the 2nd line of Page 4, $\mathcal{D}_s$ is sampled from the training dataset. The pretrained network has seen the $\mathcal{D}_s$ samples during its pretraining and potentially have lower entropy on those samples. Following the motivation of this work that targets to use unlabeled data, it is important to use unseen unlabeled data from source distribution for the auxiliary tasks, not the samples from the seen training set.

Author response:
**Regarding the use of unseen unlabeled data from the source distribution**: We thank the reviewer for this great comment. Note that as mentioned in the footnote in page 6 that we also experimented with $\mathcal D_s$ being the validation set of ImageNet and did not notice any changes in our results. For completeness, we are attaching the results below. We report the results on ImageNet-C under episodic evaluation.
|   Split    | Gauss | Shot | Impul | Defoc | Glass | Motion | Zoom | Snow | Frost | Fog | Bright | Contr | Elastic | Pixel | Jpeg | Avg.  |
|-------|-------|------|-------|-------|-------|--------|------|------|-------|-----|--------|-------|---------|-------|------|-------|
| Tain  | 62.3  | 60.1 | 60.8  | 65.3  | 65.4  | 50.3   | 46.1 | 46.8 | 53.1  | 38.7| 31.7   | 53.5  | 40.7    | 38.2  | 43.5 | 50.4  |
| Val   | 62.2  | 59.9 | 60.6  | 65.3  | 65.3  | 50.4   | 46.2 | 46.6 | 53.1  | 38.7| 31.7   | 53.2  | 40.8    | 38.1  | 43.5 | 50.4  |

Reviewer comment (Experiments):
No ablation study on the size of unlabeled source dataset $\mathcal{D}_s$ is provided. This ablation study is important as this dataset shown to guide the adaptation process.

Author response:
**Regarding the effect of the size of unlabeled source data stored on performance**: We thank the reviewer for suggesting this experimental ablation. Based on the reviewer’s suggestion, we conducted experiments where we reduced the size of source data used for DISTA. The Table below shows the results on episodic evaluation on ImageNet-C where DISTA leverages different fractions of the validation set of ImageNet sampled uniformly at random (without labels).

| Method (Fraction)       	| Gauss 	| Shot  	| Impul 	| Defoc	| Glass	| Motion   | Zoom   | Snow   | Frost | Fog	| Bright   | Contr  | Elastic | Pixel  | Jpeg   | Avg.   |
|-----------------------------|:---------:|:---------:|:---------:|:--------:|:--------:|:--------:|:------:|:------:|:-----:|:------:|:--------:|:------:|:-------:|:------:|:------:|:--------:|
| EATA(0.0\%)             	| 64.0  	| 62.1  	| 62.5  	| 66.9 	| 66.9 	| 52.5 	| 47.4   | 48.2   | 54.2  | 40.2   | 32.2 	| 54.6   | 42.2	| 39.2   | 44.7   | 51.9 	|
| DISTA(1.0\%)            	| 63.1  	| 61.1  	| 61.1  	| 66.7 	| 65.8 	| 50.9 	| 46.7   | 47.3   | 53.7  | 39.1   | 31.9 	| 54.1   | 41.5	| 38.6   | 44.1   | 51.1 	|
| DISTA(2.5\%)            	| 62.6  	| 60.8  	| 60.9  	| 65.7 	| 65.8 	| 50.9 	| 46.6   | 47.2   | 53.4  | 39.1   | 31.7 	| 54.0   | 41.5	| 38.7   | 43.8   | 50.8 	|
| DISTA(5.0\%)            	| 62.4  	| 60.4  	| 60.9  	| 65.5 	| 66.0 	| 50.5 	| 46.3   | 46.9   | 53.2  | 38.9   | 31.8 	| 53.6   | 41.0	| 38.3   | 43.8   | 50.6 	|
| DISTA(7.5\%)            	| 62.6  	| 60.3  	| 60.8  	| 65.4 	| 65.3 	| 50.4 	| 46.4   | 46.8   | 53.3  | 38.9   | 31.7 	| 53.8   | 41.2	| 38.2   | 43.7   | 50.6 	|
| DISTA(10\%)             	| 62.4  	| 60.3  	| 60.2  	| 65.5 	| 65.5 	| 50.6 	| 46.3   | 46.7   | 53.1  | 38.8   | 31.7 	| 53.5   | 41.1	| 38.2   | 43.8   | 50.5 	|
| DISTA(25\%)             	| 62.2  	| 60.4  	| 60.6  	| 65.8 	| 65.5 	| 50.5 	| 46.3   | 46.7   | 53.1  | 38.6   | 31.7 	| 53.3   | 40.9	| 38.2   | 43.6   | 50.5 	|
| DISTA(50\%)             	| 62.3  	| 60.4  	| 60.4  	| 65.1 	| 65.7 	| 50.6 	| 46.2   | 46.7   | 53.3  | 38.7   | 31.7 	| 53.2   | 40.9	| 38.3   | 43.4   | 50.5 	|
| DISTA(75\%)             	| 62.3  	| 59.9  	| 60.5  	| 64.8 	| 65.2 	| 50.4 	| 46.0   | 46.8   | 53.1  | 38.7   | 31.7 	| 53.7   | 40.9	| 38.1   | 43.5   | 50.4 	|
| DISTA(100\%)            	| 62.2  	| 59.9  	| 60.6  	| 65.3 	| 65.3 	| 50.4 	| 46.2   | 46.6   | 53.1  | 38.7   | 31.7 	| 53.2   | 40.8	| 38.1   | 43.5   | 50.4 	|

We observe that DISTA is very robots against the size of $\mathcal D_s$. In particular, we find that even with 10\% of the validation set (i.e. storing 5000 unlabeled images), DISTA improves over EATA by 1.4\% on average across all corruptions. Furthermore, with even (1\%) of source data, DISTA improves over EATA by 1\% on shot and impulse noise. This table along with its discussion is now at Appendix B.4.

Reviewer comment (Experiments):
Category-wise federated TTA, where all clients are assumed to know beforehand that adaptation carries out on similar category of domain shifts, is a controlled setting and not a realistic one. Federated TTA with diverse distribution shifts across clients would be an interesting scenario.

Author response:
**Regarding the federated evaluation**: We thank the reviewer for this remark. We see three main federated adaptation scenarios: (1) All clients experience the same distribution shift (e.g. fog). This setup would amount to a straight-forward application of federated learning methods to TTA, and thus is not the most interesting. (2) Different clients observe completely different distribution shifts, which is the most challenging setting; (3) Different clients experience different domain shifts that share some similarities or, as in our experiments, come from the same category (“setup” in our paper). We are tackling this latter scenario as a first step towards the most general case in (2). Besides being a relevant stepping stone towards more challenging federated settings, scenario (3) is also interesting in and of itself and allowed us to demonstrate that federated averaging is still useful even when clients experience different domain shifts. Since in TTA performance is measured in an online manner, the information shared via federated updates can significantly speed up adaptation.

Reviewer comment (Experiments):
What should be the size of $\mathcal{D}_s$? How the results vary with the $\mathcal{D}_s$ size? Do the samples in $\mathcal{D}_s$ need to be class-balanced?

Author response:
**Regarding the effect of the size of unlabeled source data stored on performance**: We thank the reviewer for suggesting this experimental ablation. Based on the reviewer’s suggestion, we conducted experiments where we reduced the size of source data used for DISTA. The Table below shows the results on episodic evaluation on ImageNet-C where DISTA leverages different fractions of the validation set of ImageNet sampled uniformly at random (without labels).

| Method (Fraction)       	| Gauss 	| Shot  	| Impul 	| Defoc	| Glass	| Motion   | Zoom   | Snow   | Frost | Fog	| Bright   | Contr  | Elastic | Pixel  | Jpeg   | Avg.   |
|-----------------------------|:---------:|:---------:|:---------:|:--------:|:--------:|:--------:|:------:|:------:|:-----:|:------:|:--------:|:------:|:-------:|:------:|:------:|:--------:|
| EATA(0.0\%)             	| 64.0  	| 62.1  	| 62.5  	| 66.9 	| 66.9 	| 52.5 	| 47.4   | 48.2   | 54.2  | 40.2   | 32.2 	| 54.6   | 42.2	| 39.2   | 44.7   | 51.9 	|
| DISTA(1.0\%)            	| 63.1  	| 61.1  	| 61.1  	| 66.7 	| 65.8 	| 50.9 	| 46.7   | 47.3   | 53.7  | 39.1   | 31.9 	| 54.1   | 41.5	| 38.6   | 44.1   | 51.1 	|
| DISTA(2.5\%)            	| 62.6  	| 60.8  	| 60.9  	| 65.7 	| 65.8 	| 50.9 	| 46.6   | 47.2   | 53.4  | 39.1   | 31.7 	| 54.0   | 41.5	| 38.7   | 43.8   | 50.8 	|
| DISTA(5.0\%)            	| 62.4  	| 60.4  	| 60.9  	| 65.5 	| 66.0 	| 50.5 	| 46.3   | 46.9   | 53.2  | 38.9   | 31.8 	| 53.6   | 41.0	| 38.3   | 43.8   | 50.6 	|
| DISTA(7.5\%)            	| 62.6  	| 60.3  	| 60.8  	| 65.4 	| 65.3 	| 50.4 	| 46.4   | 46.8   | 53.3  | 38.9   | 31.7 	| 53.8   | 41.2	| 38.2   | 43.7   | 50.6 	|
| DISTA(10\%)             	| 62.4  	| 60.3  	| 60.2  	| 65.5 	| 65.5 	| 50.6 	| 46.3   | 46.7   | 53.1  | 38.8   | 31.7 	| 53.5   | 41.1	| 38.2   | 43.8   | 50.5 	|
| DISTA(25\%)             	| 62.2  	| 60.4  	| 60.6  	| 65.8 	| 65.5 	| 50.5 	| 46.3   | 46.7   | 53.1  | 38.6   | 31.7 	| 53.3   | 40.9	| 38.2   | 43.6   | 50.5 	|
| DISTA(50\%)             	| 62.3  	| 60.4  	| 60.4  	| 65.1 	| 65.7 	| 50.6 	| 46.2   | 46.7   | 53.3  | 38.7   | 31.7 	| 53.2   | 40.9	| 38.3   | 43.4   | 50.5 	|
| DISTA(75\%)             	| 62.3  	| 59.9  	| 60.5  	| 64.8 	| 65.2 	| 50.4 	| 46.0   | 46.8   | 53.1  | 38.7   | 31.7 	| 53.7   | 40.9	| 38.1   | 43.5   | 50.4 	|
| DISTA(100\%)            	| 62.2  	| 59.9  	| 60.6  	| 65.3 	| 65.3 	| 50.4 	| 46.2   | 46.6   | 53.1  | 38.7   | 31.7 	| 53.2   | 40.8	| 38.1   | 43.5   | 50.4 	|

We observe that DISTA is very robots against the size of $\mathcal D_s$. In particular, we find that even with 10\% of the validation set (i.e. storing 5000 unlabeled images), DISTA improves over EATA by 1.4\% on average across all corruptions. Furthermore, with even (1\%) of source data, DISTA improves over EATA by 1\% on shot and impulse noise. This table along with its discussion is now at Appendix B.4.

Reviewer comment (Writing):
Is $f_\theta$ in eq. (3) defaults to $f_{\theta}$ at time t?

Author response:
**Is $f_theta$ in eq. (3) defaults to $f_theta$  at time $t$?** In the first part of Equation 3, f_{\theta_t} is used to estimate the entropy, while in the right hand side, $f_{\theta_t^c}$ is used. We added a clarification description about Equation 3 in page 3.

Reviewer comment (Theory):
What is the intuition behind updating ${\theta_t}^c$ using the gradients w.r.t  $\theta$ in eq. (3)?

Author response:
**Regarding the intuition behind Equation (3)**: The main intuition behind Equation (3) is to solve the optimization problem in Equation (2) in an alternating optimization manner. This is because we want to analyze the impact of the auxiliary task (second term in Equation 2), and conducting this alternating optimization approach grant us an access to the intermediate model parameters $\theta_t^c$ and allows us to compare the performances of $\theta_{t+1}$ and $\theta_t^c$. We edited the paragraph before equation (3) to clarify that. Note that the expectations in equation (3) are taken over different data distributions (resp. corrupted and clean data distributions), and while the gradients w.r.t. $\theta$ are used to update $\theta^c_t$ to get $\theta_{t+1}$, the gradient is evaluated at $\theta_t^c$. That is, 

$\theta_t^c = \theta_t - \gamma \nabla_\theta [E(f_\theta(x_t)]|_{\theta = \theta_t}$

$\theta_{t+1} = \theta_t^c - \gamma \nabla_\theta [E(f_\theta(x_t)]|_{\theta = \theta_t^c}$

We hope that this has clarified the intuition behind Equation (3).

Reviewer comment (Experiments):
Since $\mathcal{D}_s$ are sampled from training set, how the results would change if ground-truth labels are used in eq. (5)?

Author response:
**Regarding the use of labeled data instead of distillation loss**: as per the reviewers request, we conducted experiments on DISTA with replacing the distillation loss with a cross entropy loss with the ground truth labels. The overall objective function for DISTA-Labeled is:

$
\min_{\theta}  \underset{x_t\sim \mathcal S}{\mathbb E} \lambda_t(x_t) E\left(f_\theta(x_t)\right)  +   \underset{(x_s, y_s) \sim  \mathcal D_s}{\mathbb E} \lambda_s(x_s) CE\left(f_\theta(x_s),y_s\right)
$

Our results on ImageNet-C benchmark are in the table below. 

|            	| Gauss | Shot  | Impul | Defoc | Glass | Motion| Zoom  | Snow  | Frost | Fog   | Bright| Contr | Elastic | Pixel | Jpeg  | Avg.  |
|----------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|---------|-------|-------|---------|
| EATA       	| 64.0  | 62.1  | 62.5  | 66.9  | 66.9  | 52.5  | 47.4  | 48.2  | 54.2  | 40.2  | 32.2  | 54.6  | 42.2	| 39.2  | 44.7  | 51.9	|
| DISTA + Labels | 62.7  | 60.9  | 60.9  | 66.0  | 66.1  | 50.7  | 46.9  | 47.4  | 53.6  | 39.2  | 31.9  | 54.9  | 41.5	| 38.6  | 44.2  | 51.0	|
| DISTA      	| 62.2  | 59.9  | 60.6  | 65.3  | 65.3  | 50.4  | 46.2  | 46.6  | 53.1  | 38.7  | 31.7  | 53.2  | 40.8	| 38.1  | 43.5  | 50.4	|

We observe that incorporating labeled data in DISTA instead of our distillation loss does not improve its results over using an unsupervised objective (while still providing state-of-the-art results compared to other baselines in our paper). We included this experiment in Appendix B.5 and referenced it in the main paper.

Reviewer comment (Presentation):
Please mention the network architecture used in respective figure and table captions.

Author response:
**Regarding mentioning the network architecture in captions**: as per the reviewers request, we updated the captions of all tables in the main paper and appendix to mention the network architecture used.

Reviewer comment (Novelty):
The authors essentially re-invented domain adaptation, and try to re-sell this as a SOTA TTA approach.

Author response:
**Regarding the relation between our work and unsupervised domain adaptation**: We appreciate the reviewer’s comment and the interesting link to unsupervised domain adaptation, however, we disagree about casting our work as a reinvention of domain adaptation. We would like to clarify first the following:
Differences between TTA and Unsupervised Domain Adaptation: The reviewer pointed out that the difference between test-time adaptatio and unsupervised domain adaptation is the access to source data during adaptation; specifically that in TTA source data is assumed to be unavailable. We respectfully disagree with that view, and in fact we are not the first to propose a TTA method that leverages source data. RMT [A], EATA [B] and ActMAD [C] were developed under the premise that source data is available and still are considered integral parts of the TTA literature. More recently, Kang et.al. [D] distilled source data for the sake of adaptation, as mentioned in our related work. All these examples demonstrate that the use of source data is accepted and not uncommon in the Test-Time Adaptation literature. While we do agree that there are similarities between TTA and UDA, we believe there are key differences in (1) the way the pretrained model is accessing the target distribution(s), and (2) the way an adaptation method is evaluated, as we elaborate below.
(1) In Domain adaptation, the pretrained model can access the target domain *all at once* for adaptation. However, in Test-Time Adaptation, the pretrained model accesses the target domain as a stream of data. The model observes each revealed batch of data only *once* unlike domain adaptation where one could potentially revisit examples from the target domain multiple times. 
(2) The performance in domain adaptation is calculated *after* adaptation. That is, after a pretrained model adapts on a target domain, its performance on novel samples from the target domain is calculated. In Test-Time Adaptation, the performance of an adaptation method is calculated in an *online* manner where the model needs to provide the prediction of a given batch before receiving the next batch from the stream (please refer to the preliminary paragraph in our method section where we formalize the interaction between a TTA method and the stream of unlabeled data).
Given these differences, we believe that our work, similar to [A, B, C, D], falls into the test time adaptation category where we followed the standard evaluation setups in the literature.
While we do believe that cross-talk between the two fields is very interesting and important, we think that this deserves a work on its own and it falls outside the scope of our submission.Nonetheless, we are happy to include a discussion and comparison (if possible) on related unsupervised domain adaptation methods that the reviewer thinks are most related.

[A] Robust Mean Teacher for Continual and Gradual Test-Time Adaptation, CVPR 2023

[B] Efficient Test-Time Model Adaptation without Forgetting, ICML 2022

[C] ActMAD: Activation Matching To Align Distributions for Test-Time-Training, CVPR2023

[D] Leveraging proxy of training data for test-time adaptation, ICML 2023.

Reviewer comment (Experiments):
The paper positions itself in the space of test-time adaptation, which is exactly the setting in which source data is not available. The fact that the SOTA TTA model, EATA*, when evaluated in an unsupervised domain adaptation setting instead of a TTA setting becomes better might be obvious.

Author response:
**Regarding the relation between our work and unsupervised domain adaptation**: We appreciate the reviewer’s comment and the interesting link to unsupervised domain adaptation, however, we disagree about casting our work as a reinvention of domain adaptation. We would like to clarify first the following:
Differences between TTA and Unsupervised Domain Adaptation: The reviewer pointed out that the difference between test-time adaptatio and unsupervised domain adaptation is the access to source data during adaptation; specifically that in TTA source data is assumed to be unavailable. We respectfully disagree with that view, and in fact we are not the first to propose a TTA method that leverages source data. RMT [A], EATA [B] and ActMAD [C] were developed under the premise that source data is available and still are considered integral parts of the TTA literature. More recently, Kang et.al. [D] distilled source data for the sake of adaptation, as mentioned in our related work. All these examples demonstrate that the use of source data is accepted and not uncommon in the Test-Time Adaptation literature. While we do agree that there are similarities between TTA and UDA, we believe there are key differences in (1) the way the pretrained model is accessing the target distribution(s), and (2) the way an adaptation method is evaluated, as we elaborate below.
(1) In Domain adaptation, the pretrained model can access the target domain *all at once* for adaptation. However, in Test-Time Adaptation, the pretrained model accesses the target domain as a stream of data. The model observes each revealed batch of data only *once* unlike domain adaptation where one could potentially revisit examples from the target domain multiple times. 
(2) The performance in domain adaptation is calculated *after* adaptation. That is, after a pretrained model adapts on a target domain, its performance on novel samples from the target domain is calculated. In Test-Time Adaptation, the performance of an adaptation method is calculated in an *online* manner where the model needs to provide the prediction of a given batch before receiving the next batch from the stream (please refer to the preliminary paragraph in our method section where we formalize the interaction between a TTA method and the stream of unlabeled data).
Given these differences, we believe that our work, similar to [A, B, C, D], falls into the test time adaptation category where we followed the standard evaluation setups in the literature.
While we do believe that cross-talk between the two fields is very interesting and important, we think that this deserves a work on its own and it falls outside the scope of our submission.Nonetheless, we are happy to include a discussion and comparison (if possible) on related unsupervised domain adaptation methods that the reviewer thinks are most related.

[A] Robust Mean Teacher for Continual and Gradual Test-Time Adaptation, CVPR 2023

[B] Efficient Test-Time Model Adaptation without Forgetting, ICML 2022

[C] ActMAD: Activation Matching To Align Distributions for Test-Time-Training, CVPR2023

[D] Leveraging proxy of training data for test-time adaptation, ICML 2023.

Reviewer comment (Novelty):
The potential value of a proper evaluation on applying domain adaptation techniques to TTA—potentially increasing cross-talk between the two fields/evaluation settings—is dismissed by how the paper is written. There is no mention (besides a few titles in the bibliography) of the term "domain adaptation" or "unsupervised domain adaptation" in the main paper.

Author response:
**Regarding the relation between our work and unsupervised domain adaptation**: We appreciate the reviewer’s comment and the interesting link to unsupervised domain adaptation, however, we disagree about casting our work as a reinvention of domain adaptation. We would like to clarify first the following:
Differences between TTA and Unsupervised Domain Adaptation: The reviewer pointed out that the difference between test-time adaptatio and unsupervised domain adaptation is the access to source data during adaptation; specifically that in TTA source data is assumed to be unavailable. We respectfully disagree with that view, and in fact we are not the first to propose a TTA method that leverages source data. RMT [A], EATA [B] and ActMAD [C] were developed under the premise that source data is available and still are considered integral parts of the TTA literature. More recently, Kang et.al. [D] distilled source data for the sake of adaptation, as mentioned in our related work. All these examples demonstrate that the use of source data is accepted and not uncommon in the Test-Time Adaptation literature. While we do agree that there are similarities between TTA and UDA, we believe there are key differences in (1) the way the pretrained model is accessing the target distribution(s), and (2) the way an adaptation method is evaluated, as we elaborate below.
(1) In Domain adaptation, the pretrained model can access the target domain *all at once* for adaptation. However, in Test-Time Adaptation, the pretrained model accesses the target domain as a stream of data. The model observes each revealed batch of data only *once* unlike domain adaptation where one could potentially revisit examples from the target domain multiple times. 
(2) The performance in domain adaptation is calculated *after* adaptation. That is, after a pretrained model adapts on a target domain, its performance on novel samples from the target domain is calculated. In Test-Time Adaptation, the performance of an adaptation method is calculated in an *online* manner where the model needs to provide the prediction of a given batch before receiving the next batch from the stream (please refer to the preliminary paragraph in our method section where we formalize the interaction between a TTA method and the stream of unlabeled data).
Given these differences, we believe that our work, similar to [A, B, C, D], falls into the test time adaptation category where we followed the standard evaluation setups in the literature.
While we do believe that cross-talk between the two fields is very interesting and important, we think that this deserves a work on its own and it falls outside the scope of our submission.Nonetheless, we are happy to include a discussion and comparison (if possible) on related unsupervised domain adaptation methods that the reviewer thinks are most related.

[A] Robust Mean Teacher for Continual and Gradual Test-Time Adaptation, CVPR 2023

[B] Efficient Test-Time Model Adaptation without Forgetting, ICML 2022

[C] ActMAD: Activation Matching To Align Distributions for Test-Time-Training, CVPR2023

[D] Leveraging proxy of training data for test-time adaptation, ICML 2023.

Reviewer comment (Evaluation):
There is no discussion on the additional memory burden introduced by storing the source data. In Figure 2, EATA and DISTA are compared in terms of computation but not memory. The source dataset requirement is memory heavy: Using the full ImageNet dataset would add 160GB of storage, which could alternatively be used to run much larger models offering more drastic performance gains than DISTA. The memory requirement is a major limitation of the method.

Author response:
**Regarding the discussion on memory burden**: We would like to thank the reviewer for this comment. We would like to point out that DISTA uses the same amount of source data as EATA uses for calculating the Fisher matrix. In particular, DISTA leverages a subset of (50,000) unlabeled images which translates to around 4% of ImageNet training set in size. Nevertheless, we do agree that the performance improvement of DISTA comes at an additional storage cost, similar to [A, B, C, D]. We have added a limitation section in our appendix discussing the additional memory requirements of DISTA. Further, we conducted experiments where we use subsample the ImageNet validation set (unlabeled) and use it as  $\mathcal D_s$ to reduce the memory burden. Here are the results:
| Method (Fraction)       	| Gauss 	| Shot  	| Impul 	| Defoc	| Glass	| Motion   | Zoom   | Snow   | Frost | Fog	| Bright   | Contr  | Elastic | Pixel  | Jpeg   | Avg.   |
|-----------------------------|:---------:|:---------:|:---------:|:--------:|:--------:|:--------:|:------:|:------:|:-----:|:------:|:--------:|:------:|:-------:|:------:|:------:|:--------:|
| EATA(0.0\%)             	| 64.0  	| 62.1  	| 62.5  	| 66.9 	| 66.9 	| 52.5 	| 47.4   | 48.2   | 54.2  | 40.2   | 32.2 	| 54.6   | 42.2	| 39.2   | 44.7   | 51.9 	|
| DISTA(1.0\%)            	| 63.1  	| 61.1  	| 61.1  	| 66.7 	| 65.8 	| 50.9 	| 46.7   | 47.3   | 53.7  | 39.1   | 31.9 	| 54.1   | 41.5	| 38.6   | 44.1   | 51.1 	|
| DISTA(2.5\%)            	| 62.6  	| 60.8  	| 60.9  	| 65.7 	| 65.8 	| 50.9 	| 46.6   | 47.2   | 53.4  | 39.1   | 31.7 	| 54.0   | 41.5	| 38.7   | 43.8   | 50.8 	|
| DISTA(5.0\%)            	| 62.4  	| 60.4  	| 60.9  	| 65.5 	| 66.0 	| 50.5 	| 46.3   | 46.9   | 53.2  | 38.9   | 31.8 	| 53.6   | 41.0	| 38.3   | 43.8   | 50.6 	|
| DISTA(7.5\%)            	| 62.6  	| 60.3  	| 60.8  	| 65.4 	| 65.3 	| 50.4 	| 46.4   | 46.8   | 53.3  | 38.9   | 31.7 	| 53.8   | 41.2	| 38.2   | 43.7   | 50.6 	|
| DISTA(10\%)             	| 62.4  	| 60.3  	| 60.2  	| 65.5 	| 65.5 	| 50.6 	| 46.3   | 46.7   | 53.1  | 38.8   | 31.7 	| 53.5   | 41.1	| 38.2   | 43.8   | 50.5 	|
| DISTA(25\%)             	| 62.2  	| 60.4  	| 60.6  	| 65.8 	| 65.5 	| 50.5 	| 46.3   | 46.7   | 53.1  | 38.6   | 31.7 	| 53.3   | 40.9	| 38.2   | 43.6   | 50.5 	|
| DISTA(50\%)             	| 62.3  	| 60.4  	| 60.4  	| 65.1 	| 65.7 	| 50.6 	| 46.2   | 46.7   | 53.3  | 38.7   | 31.7 	| 53.2   | 40.9	| 38.3   | 43.4   | 50.5 	|
| DISTA(75\%)             	| 62.3  	| 59.9  	| 60.5  	| 64.8 	| 65.2 	| 50.4 	| 46.0   | 46.8   | 53.1  | 38.7   | 31.7 	| 53.7   | 40.9	| 38.1   | 43.5   | 50.4 	|
| DISTA(100\%)            	| 62.2  	| 59.9  	| 60.6  	| 65.3 	| 65.3 	| 50.4 	| 46.2   | 46.6   | 53.1  | 38.7   | 31.7 	| 53.2   | 40.8	| 38.1   | 43.5   | 50.4 	|

We observe that  even with 10\% of the validation set (\emph{i.e.} storing 5000 unlabeled images), DISTA improves over EATA by 1.4\% on average across all corruptions. Furthermore, with even (1\%) of source data, DISTA improves over EATA by 1\% on shot and impulse noise. We added this experiment with its discussion in Table 12 (Appendix B.4).
Above all,, it is noteworthy that the cost of storage is generally much cheaper than the cost of computation. For instance, based on Table 3 in [E] (Appendix A), storing the entire ImageNet 1K dataset costs 66 cents which is much cheaper than training a single model on the same dataset (cost of 500 USD).  That is, storing more data is cheaper than leveraging larger models for inference.


[E] Computationally Budgeted Continual Learning: What Does Matter?, CVPR2023

Reviewer comment (Experiments):
The paper does not consider datasets for natural corruptions such as ImageNet-R, ImageNet-D, or ObjectNet in addition to ImageNet-C.

Author response:
**Regarding conducting experiments on more datasets**: We thank the reviewer for this suggestion. In our paper, we followed the common practice in the TTA literature [A, B, F, G]  and  conducted experiments on the two largest  TTA benchmarks: ImageNet-C and ImageNet-3DCC

[F] Tent: Fully Test-time Adaptation by Entropy Minimization, ICLR 2021

[G] Towards Stable Test-Time Adaptation in Dynamic Wild World, ICLR 2023

Reviewer comment (Presentation):
The table headings do not specify which model is used for the results.

Author response:
**Regarding mentioning the network architecture in captions**: We would like to thank the reviewer for this comment. We have modified the table captions to include the model used in the experiments.

Reviewer comment (Evaluation):
The stochasticity of the method is unclear; error bars should be provided.

Author response:
**Regarding the stochasticity of DISTA**: We have repeated the experiments with DISTA (episodic evaluation on Gaussian Noise in ImageNet-C) 10 times, where we found that the maximum difference between two runs (in terms of error rate) is 0.1%. This shows that our method provides consistent results with very negligible stochasticity. We note that this stochasticity is due to either different order of data revealed from the stream (as pointed out in our second distinction between UDA and TTA) and the random sampling from $\mathcal D_s$.

Reviewer comment (Reproducibility):
It is unclear whether the results for other methods and the proposed method were re-ran within the same codebase or copied from other papers.

Author response:
**Regarding rerunning baselines or copying numbers from original papers**: We re-evaluated the previous method and reproduced their reported results in the original paper. As mentioned in Appendix B.1, we used the original publicly available code for each method.

Reviewer comment (Experiments):
There is no explanation of how the parameters $\epsilon$ and $E_0$, which influence the weighting of both losses, were tuned. An analysis table for varying these parameters is missing.

Author response:
**Regarding the effect of $\epsilon$ and $E_0$**: We appreciate the reviewer’s comment. We note here that  Both $E_0$ and $\epsilon$ are set following EATA with their recommended hyperparameters. Further,  we conducted  experiments with two values of E_0 and report the results in the table below.
| E0   | Gauss     	| Shot      	| Impul     	| Defoc     	| Glass     	| Motion    	| Zoom      	| Snow      	| Frost     	| Fog 	| Bright| Contr | Elastic   	| Pixel     	| Jpeg      	| Avg.      	|
|----------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|
| 0.4 log(1000)   | 62.9      	| 60.7      	| 61.4 | 65.8      	| 65.9      	| 51.2      	| 46.5      	| 47.6      	| 53.7      	| 39.3      	| 31.7      	| 54.3      	| 41.6      	| 38.5      	| 44.1      	| 51.0      	|
| 0.5 log(1000)	| 62.2      	| 59.9      	| 60.6      	| 65.3      	| 65.3      	| 50.4      	| 46.2      	| 46.6      	| 53.1      	| 38.7      	| 31.7      	| 53.2      	| 40.8      	| 38.1      	| 43.5	| 50.4

We observe that varying E_0 results in performance variation due to filtering more/less samples from the update step. This result is in Table 11 in the Appendix.

Reviewer comment (Writing):
In the abstract, the claim that a "key challenge in TTA" is adapting on limited data requires references. In many continual settings, test-time models have access to a wealth of data.

Author response:
**Regarding ‘limited data’ in our abstract**: We thank the reviewer for this comment. We would like to clarify here that we meant by limited data is the amount of data the stream reveals at a given time step $t$. TTA methods can only leverage this amount of data (which is limited by construction) for adaptation (our first distinction between TTA and UDA). We agree that a wealth of data exists, but it is made available in a sequential manner in test-time adaptation.

Reviewer comment (Experiments):
In the abstract, the claim that "conducting several adaptation steps ... will lead to overfitting" requires references or experimental evidence within the paper showing that DISTA resolves this.

Author response:
**Regarding "conducting several adaptation steps ... will lead to overfitting"**: We appreciate this comment. We note here that since the stream $\mathcal S$ reveals data in batches for the adaptation method, conducting several adaptation steps will make the model overfit to the given batch as all the adaptation steps are performed on the same batch. To confirm this claim, we conducted an experiment with the previous state-of-the-art method EATA where we allow the model to adapt with 1, 2, 3, 4, 5 steps for each received batch from the stream. Results are reported in the table below:

| Number of steps  | Gauss | Shot | Impul | Defoc | Glass | Motion | Zoom | Snow | Frost | Fog  | Bright | Contr | Elastic | Pixel | Jpeg | Avg. |
|--------|-------|------|-------|-------|-------|--------|------|------|-------|------|--------|-------|---------|-------|------|--------|
| EATA-1 | 64.0  | 62.1 | 62.5  | 66.9  | 66.9  | 52.5   | 47.4 | 48.2 | 54.2  | 40.2 | 32.2   | 54.6  | 42.2	| 39.2  | 44.7 | 51.9   |
| EATA-2 | 68.3  | 63.8 | 65.9  | 72.6  | 72.4  | 53.7   | 48.6 | 49.3 | 55.7  | 40.5 | 32.9   | 58.2  | 42.9	| 39.8  | 45.7 | 54.0   |
| EATA-3 | 74.1  | 70.5 | 75.6  | 86.9  | 81.2  | 58.9   | 52.0 | 51.8 | 60.2  | 41.7 | 34.2   | 74.4  | 45.4	| 41.5  | 48.0 | 59.8   |
| EATA-4 | 90.4  | 82.1 | 85.7  | 96.8  | 91.5  | 67.1   | 52.9 | 55.5 | 67.2  | 43.0 | 35.2   | 95.7  | 46.1	| 42.0  | 49.0 | 66.7   |
| EATA-5 | 95.3  | 92.9 | 93.2  | 97.1  | 96.2  | 70.6   | 56.6 | 56.9 | 74.7  | 45.1 | 35.4   | 97.3  | 47.9	| 44.4  | 51.2 | 70.3   |

We observe that the more adaptation steps EATA conducts on each revealed batch, the worse its performance becomes. This experiment confirms our claims and demonstrates the effectiveness of our auxiliary task in improving the performance with the additional adaptation step.