Paper: Reliable Test-Time Adaptation via Agreement-on-the-Line (ICLR-2024)

Reviewer comment (Experiments):
The claim of “OOD and ID performance/agreement correlation” requires more empirical evidence. Please refer to Question 1.

Author response:
We hope our responses in [here](https://openreview.net/forum?id=iEFMwP5wng&noteId=vZZSNTMlWr) clarified the reviewer's concern about the empirical evidence of the OOD and ID performance/agreement correlation.

Reviewer comment (Theory):
The proposed methods rely on full access to the ID data and work in an offline manner, which slightly weakens its application scenarios.

Author response:
We hope our responses in [here](https://openreview.net/forum?id=iEFMwP5wng&noteId=pAkj6nKxb2) sufficiently addressed the concern about the offline nature of the proposed methods.

Reviewer comment (Experiments):
Are there any (or potential) solutions that can modify the proposed method to be an online version? In other words, how about the performance of Algorithm 1 work in an online setting?

Author response:
We appreciate the reviewer's feedback, and we are pleased to provide more details about the modifications made in our method for an online setup. The algorithms (https://tinyurl.com/5n72hev6) directly compare the online version (**above**) and the offline-version (**below**). Blue-colored comments highlight the newly added detail from the algorithm in the original submission.

A key modification is that **we apply ALine-S/D only to the predictions of current ID and OOD batch at each iteration (Line 10 in Algorithm 1) in the online version**. In this case, $\mathcal P_\text{ID}$ and $\mathcal P_\text{OOD}$ contain predictions of $n$ models on the current test batch ($x_\text{ID}$ and $x_\text{OOD}$), and ALine-S/D is applied to these. 

On the contrary, in the offline version, we apply ALine-S/D at the end of the algorithm (Line 11 in Algorithm 2) after saving predictions on the entire test set ($\mathcal X_\text{ID}$ and $\mathcal X_\text{OOD}$) in $\mathcal P_\text{ID}$ and $\mathcal P_\text{OOD}$, with ALine-S/D then applied to these aggregated predictions.

Please let us know if there are any additional questions.

Reviewer comment (Experiments):
Are there any sensitivity analyses about $n$ in Algorithm 1?

Author response:
We hope our responses in [here](https://openreview.net/forum?id=iEFMwP5wng&noteId=pAkj6nKxb2) sufficiently addressed the concern about the sensitivity of our method to the number of models $n$.

Reviewer comment (Experiments):
For Calibration and Hyper-parameter Selection, how many ID samples are needed? Will this ID sample number affect the performance significantly?

Author response:
We examined the sensitivity of our method on the number of ID samples [here](https://openreview.net/forum?id=iEFMwP5wng&noteId=3aU3nNw8Bt), and we hope this sufficiently addressed the concern.

Reviewer comment (Evaluation):
If possible, I am also curious about the ID-OOD correlation of MEMO [Memo: Test time robustness via adaptation and augmentation] and Contrastive Learning Objectives [Contrastive Test-Time Adaptation].

Author response:
We tested MEMO [1] with the same set of architectures in Figure 1 on CIFAR10-C Gaussian Noise. The results show **strong correlated lines in accuracy and agreement** (https://tinyurl.com/4tbnrsv2), confirming the consistency of AGL/ACL phenomena across different TTA baselines. The MAE (%) of accuracy estimation on models adapted with MEMO on CIFAR10-C Gaussian Noise is 1.16 % for ALine-S and 1.16 % for ALine-D.

AdaContrast [2] employs contrastive learning with online pseudo-labeling for target domains. Due to the limited time in the rebuttal, we could not implement the baseline to see if AGL/ACL persist in the method. We will make sure to include AdaContrast as one of our TTA baselines to observe the linear trends in our camera-ready version.

[1] Zhang et al., "MEMO: Test Time Robustness via Adaptation and Augmentation", NeurIPS'22

[2] Chen et al., "Contrastive Test-Time Adaptation", CVPR'22

Reviewer comment (Evaluation):
The authors have empirically observed the occurrence of agreement-on-the-line (AGL) and accuracy-on-the-line (ACL) following test-time adaptation (TTA). To enhance the manuscript, it would be beneficial for the authors to provide a more extensive explanation and discussion regarding this observed phenomenon.

Author response:
We have added detailed explanations about their definitions as well as characteristics in the related work (Section 4) in our revised manuscript.

Reviewer comment (Presentation):
Figures 1-3 are difficult to understand. For example, it is unclear what the values of the horizontal and the vertical axes in Figure 1 represent. More explanations should be provided.

Author response:
The figures mainly show the TTAed models’ accuracy (%) tested on ID (x-axis) and OOD (y-axis), represented as dots with blue color. At the same time, these models’ agreement (%) tested on ID and OOD are plotted in the same x-axis and y-axis, denoted as pink dots. In addition, we plot the blue and pink lines that represent the linear fit of each dot, corresponding to the linear correlations between ID and OOD. In Figures 1 and 2, each dot represents the accuracy/agreement of the models with different architectures, while in Figure 3, they denote that of the models adapted with different adaptation hyperparameters. We provided such detailed explanations on each caption of the figures in the revised version of our submission.

Reviewer comment (Writing):
In the Introduction, the authors claim that "Baek et al. (2022) show that the ID and OOD agreement between classifiers shows a strong linear correlation". However, Baek et al. (2022) only study the correlation of disagreement and error between classifiers. It would be preferable to provide a more precise description.

Author response:
We would like to clarify that our original claim in the Introduction was that “Baek et al. (2022)[1] show that the OOD agreement between classifiers shows a strong linear correlation with their ID agreement.” To expound on our intention, we quote the specific sentence from Baek et al.(2022)[1] that encapsulates the core observation: “ Whenever accuracy-on-the-line holds, we observe that the OOD agreement between the predictions of any two pairs of neural networks (with potentially different architectures) also observes a strong linear correlation with their ID agreement.” We clarified our claim in the Introduction (Section 1) of our revised submission.

[1] Baek et al., Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift, NeurIPS’22

Reviewer comment (Experiments):
In Figure 3, the authors demonstrate that the phenomenon of AGL and ACL also occurs when varying TTA hyperparameters, such as learning rate and batch size. However, from Table 1 to Table 3, the authors only study the effect of the proposed methods using different architectures or different training checkpoints. More ablation study using different hyperparameter setup is suggested.

Author response:
To supplement the incomplete results the reviewer pointed out, we provide the results of accuracy estimation and calibration using different hyperparameter setups, which include learning rates, adapt steps, batch sizes, and checkpoints. In the Table below, we compare the estimation results (MAE (%)) of our methods (on ETA) with other baselines on ImageNet-C (averaged over every corruption). The results show that our methods with different hyperparameters achieve state-of-the-art performances across various hyperparameters and datasets (Results on CIFAR10-C/CIFAR100-C (TENT) are provided in Section A.6 and Table 8 of the appendix in our revised submission).

**ImageNet-C [MAE (\%)]**
| Setup | ATC | DOC-feat | AC | Agreement | ALine-S | ALine-D
| ---------- | ---------- | ---------- | ---------- | ---------- | ---------- | ---------- |
| Learning rate | 5.60 | 5.28 | 37.12 | 18.22 | 4.46 | **3.70**|
| Adapt step | 11.22 | 9.31 | 31.51 | 32.60 | 6.01 | **5.75** |
| Batch size | 8.19 | 6.70 | 26.18 | 14.29 | 3.45 | **2.34** |
| Checkpoint | 2.96 | 5.77 | 12.06 | 21.28 | 3.01 | **2.70** |

We also provide the calibration results using different hyperparameters when applying TENT on CIFAR10-C and CIFAR100-C (averaged over every corruption). Similar to Table 3, our methods effectively reduce the calibration error using the adapt steps, learning rates, or batch sizes.

| Setup | CIFAR10-C | CIFAR100-C |
| ---------- | ---------- | ---------- | 
| Uncalibrated | 7.76 |13.40 |
| Learning rate | 3.23 | 3.29 |
| Adapt step | 3.40 | 4.56 |
| Batch size | 3.50 | 3.67 |

Reviewer comment (Writing):
On page 7, the sentence “let X the random variable” should be changed to “let X be the random variable”.

Author response:
We revised the grammatical error in Section 3.2 of our revised version.

Reviewer comment (Novelty):
The novelty of this method is relatively limited; it is an application of Baek et al. (2022) to the TTA setup.

Author response:
While acknowledging the insights from Baek et al. (2022), we want to emphasize that our work explores the AGL phenomenon in adaptive setups. In contrast, Baek et al. (2022) primarily focused on pretrained models with their weights unchanged during test time. We specifically identify an unexpected condition where linear trends persist or even **hold stronger**, as evidenced by the extensive results across datasets, TTA baselines, and experimental setups.

Furthermore, our proposed methods for accuracy estimation, calibration, and hyperparameter optimization, represent a novel contribution. These methods facilitate the reliable utilization of TTA, an aspect rarely explored or addressed by previous studies due to a lack of access to OOD labels.

Reviewer comment (Reproducibility):
This method requires adapting multiple models during test time at the specific distribution shift in order to compute the agreement and use it to improve TTA. This might limit practical applications where training multiple models is expensive and, furthermore, the improvements on TTA seem to be on hindsight as one first needs to train multiple models in order to understand whether TTA will yield improvement. Furthermore, given that TTA usually operates on a stream of OOD data and provides a stream of predictions, it is unclear how predictions will be made at test time; does one use the ID model, (one of) the TTA models or does one wait up until the agreement-on-the-line has been computed before making any predictions on the stream?

Author response:
We elaborate on the detailed procedure for making predictions during TTA in our method. TTA operates as it usually does: it updates model parameters online with a stream of OOD data. At each iteration, after parameter updates, we perform inference on batches of OOD and ID data, storing their predictions. After running these procedures on multiple models, we calculate their accuracy and agreement using predictions. Subsequently, we utilize ALine-S/D to estimate OOD accuracy based on the AGL/ACL. We have provided the detailed explanation on how we obtain models’ predictions in the ID and OOD during TTA in Section 3.1 of the revised submission.

Moreover, as demonstrated [here](https://openreview.net/forum?id=iEFMwP5wng&noteId=pAkj6nKxb2), our method is applicable in the online setup. To make predictions, for each iteration, we update models with streamed OOD batch and perform inference on both OOD and ID batches to obtain predictions. Instead of storing predictions until the entire test set, we calculate accuracy and agreement on the current batch using AGL/ACL and apply our methods for accuracy estimation and calibration.

Reviewer comment (Theory):
What is considered to be \(h(x)\)? Is it the most probable class under \(h\)? If so, why not define agreement in terms of the entire distribution on the output space for models, e.g., some kind of expected divergence between \(h(x)\) and \(h'(x)\)?

Author response:
As noted by the reviewer, h(x) denotes the most probable class predicted by the model h given data x. We clarified the definition of h(x) in Section 2.1 of the revised version of our submission.

We want to clarify that in our work, we employed the definition of agreement by following prior studies that have established the concept of (dis)agreement between classifiers to assess their generalizations in both in-domain contexts [1,2,3] and OOD scenarios [4]. These studies empirically showcased the superiority of assessing generalizations of the models via (dis)agreement between models trained in different setups. A recent paper [5] attempted to demystify the effectiveness of employing such (dis)agreement between models for predicting OOD performances within a theoretical framework.

[1] Madani et al., “Co-Validation: Using Model Disagreement on Unlabeled Data to Validate Classification Algorithms”, NIPS’04

[2] Nakkiran and Bansal, “Distributional generalization: A new kind of generalization”, Arxiv’21

[3] Jiang et al., “Assessing generalization of SGD via disagreement”, ICLR’22

[4] Baek et al., Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift, NeurIPS’22

[5] Lee et al., “Demystifying Disagreement-on-the-Line in High Dimensions”, ICML’23

Reviewer comment (Reproducibility):
How many models were used to get the agreement / accuracy lines at the figures?

Author response:
Here’s the detailed setups for the figures:
* Figures 1,2: a total of 11 and 9 models of different architectures for CIFAR10-C, and ImageNet-C, respectively. 
* Figure 3 first row: a total of 10 models of different learning rates, five of adapt steps, 9 of batch sizes, and 10 of checkpoints in CIFAR10-C
* Figure 3 second row: a total of 7 models of different learning rates, five of adapt steps, 8 of batch sizes and 10 of checkpoints in ImageNet-C

Reviewer comment (Experiments):
At algorithm 1, it seems that predictions on the ID and OOD data are done under a model that is continuously updated on OOD data, therefore, the ordering of the batches might have an effect on the agreement lines. Do the results change significantly under different random seeds?

Author response:
To answer this, we iterated over 10 different random seeds to run TENT on different ordering of batches. Specifically, we tested in the identical setup of Figure 1 of our submission, where we applied TENT on 11 models with different architectures, evaluated on CIFAR10-C Gaussian Noise. In table below, we reported the mean and standard deviation of the AGL/ACL across different orderings, $R^2$, slope and bias of the linear line.

We noticed that strong AGL/ACL consistently hold across varying data orders, maintaining consistently high $R^2$ as well as (almost) the same slope and bias between accuracy and agreement correlation lines. Thus, our method estimates their accuracies with consistently low MAE (%), which are **1.67±0.05** and **1.70±0.05**, for ALine-S and ALine-D, respectively. This demonstrates that our method’s superior accuracy estimation is not affected by the different order of batches.

**CIFAR10-C**
| Method | $R^2$ (AGL) | $R^2$ (ACL) | Slope (AGL) | Slope (ACL) | Bias (AGL) | Bias (ACL) |
| ---------- | ---------- | ---------- | ---------- | ---------- |---------- | ---------- |
| TENT | 0.90±0.3 | 0.86±0.04 | 1.05±0.06| 0.96±0.04 | -0.53±0.07 | -0.40±0.05|