Paper: Prototypical Influence Function for Fully Test-time Adaptation (ICLR-2024)

Reviewer comment (Theory):
The PIF regularizer approximates the model-trusting data with the weight prototypes. It is confusing whether the weight prototypes can always be positively associated with the unseen model-trusting data during testing. Considering the distribution shift between training and test data, it might be more convincing to explain under what conditions the correlation between the weight prototypes and the unseen model-trusting data can guarantee the accurate approximation of Eq. (4).

Author response:
We define model-trusting data as virtual data that produces near one-hot predictions. Considering how the last layer operates, for near one-hot predictions, it needs to align well with the last layer weights. Thus, we can consider the last layer weight vector itself as the prototype for model-trusting data. For effective use of model-trusting data, there should not be a significant distribution shift between source and target data, and the model should share the learned embedding space.

Reviewer comment (Theory):
The computational complexity of Algorithm 1 is not analyzed, especially for the step of updating parameters of the BN layer using Eq. (11). More specifically, what is the complexity of the PIF regularizer with respect to the parameters of BN layers?

Author response:
The PIF value can be computed using only forward pass values, and its complexity concerning BN layer parameters is the same as that of EM loss..

Reviewer comment (Reproducibility):
The selection of hyperparameters is unclear. The experiments show that different values of $\gamma$ and $\beta$ are used in different data sets. However, it is not explained how these hyperparameters are selected in the test-time adaptation settings.

Author response:
We performed a greedy search for optimal hyperparameters.

Reviewer comment (Theory):
It is unclear why the weighting strategy is applied for deriving the approximation in Eq. (8).

Author response:
The weighting strategy in designing the PIF regularizer is crucial as it determines which class's prototype gradient to align with, given the absence of labels for test data.

Reviewer comment (Experiments):
From the implementation details it looks like the method requires specific parameter fine tuning per method. As in the Supplement B, for each dataset and each baseline method, different sets of parameters were used. Can we at least use one set of parameters for a dataset? Fig. 4 suggests that we can simply use large gamma parameter and achieve good results. But in the Supplement B some very small values were used. Can we see Fig. 4 for example ImageNet-C + EATA or ImageNet-3DCC + PIL? Also for SAR very large gamma value was used (larger than maximum value in Fig. 4). Can you explain here? Also there seems to be a typo in the table for CIFAR-10-C and SAR.

Author response:
**W1.** There are many details missing about the hyper-parameters:

**a)** We applied PIF to TENT, EATA, PL, and SAR for each dataset, and each method has different loss functions, making it impossible to use the same hyperparameters across methods.

For ImageNet-C + EATA, an ablation study on the hyperparameter gamma yielded the following results:

| gamma      | 10    | 20    | 30    | 40    | 50    | 60    | 70    | 80    | 90    | 100   |
|------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
| error_rate | 51.74 | 51.51 | 51.38 | 51.35 | 51.37 | 51.43 | 51.51 | 51.62 | 51.81 | 52.03 |

SAR requires a larger gamma due to applying PIF only in the first update during backpropagation.

**b)** Beta values were chosen based on experimentation. For TENT+PIF on ImageNet-C, fixing gamma at 150 and varying beta produced the following error rates:

| beta       | 0    | 0.5  | 1    | 1.5  | 2    | 3    | 4    | 5    | 6    |
|------------|------|------|------|------|------|------|------|------|------|
| error rate | 64.8 | 58.4 | 55.9 | 55.3 | 55.3 | 55.6 | 55.9 | 56.2 | 56.3 |

**c)** We performed a greedy search for optimal hyperparameters.

**d)** We use the same learning rate as the baseline.

Reviewer comment (Experiments):
A similar question arises for the Beta parameter. Beta = 0 for CIFAR and 2 for ImageNet. What happened if other parameters were used?

Author response:
**W1.** There are many details missing about the hyper-parameters:

**a)** We applied PIF to TENT, EATA, PL, and SAR for each dataset, and each method has different loss functions, making it impossible to use the same hyperparameters across methods.

For ImageNet-C + EATA, an ablation study on the hyperparameter gamma yielded the following results:

| gamma      | 10    | 20    | 30    | 40    | 50    | 60    | 70    | 80    | 90    | 100   |
|------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
| error_rate | 51.74 | 51.51 | 51.38 | 51.35 | 51.37 | 51.43 | 51.51 | 51.62 | 51.81 | 52.03 |

SAR requires a larger gamma due to applying PIF only in the first update during backpropagation.

**b)** Beta values were chosen based on experimentation. For TENT+PIF on ImageNet-C, fixing gamma at 150 and varying beta produced the following error rates:

| beta       | 0    | 0.5  | 1    | 1.5  | 2    | 3    | 4    | 5    | 6    |
|------------|------|------|------|------|------|------|------|------|------|
| error rate | 64.8 | 58.4 | 55.9 | 55.3 | 55.3 | 55.6 | 55.9 | 56.2 | 56.3 |

**c)** We performed a greedy search for optimal hyperparameters.

**d)** We use the same learning rate as the baseline.

Reviewer comment (Reproducibility):
I assume oracle hyper-parameter selection was used to compute the optimal parameters for the proposed method. Can the authors provide how the search was run, i.e., what was the range of parameters?

Author response:
**W1.** There are many details missing about the hyper-parameters:

**a)** We applied PIF to TENT, EATA, PL, and SAR for each dataset, and each method has different loss functions, making it impossible to use the same hyperparameters across methods.

For ImageNet-C + EATA, an ablation study on the hyperparameter gamma yielded the following results:

| gamma      | 10    | 20    | 30    | 40    | 50    | 60    | 70    | 80    | 90    | 100   |
|------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
| error_rate | 51.74 | 51.51 | 51.38 | 51.35 | 51.37 | 51.43 | 51.51 | 51.62 | 51.81 | 52.03 |

SAR requires a larger gamma due to applying PIF only in the first update during backpropagation.

**b)** Beta values were chosen based on experimentation. For TENT+PIF on ImageNet-C, fixing gamma at 150 and varying beta produced the following error rates:

| beta       | 0    | 0.5  | 1    | 1.5  | 2    | 3    | 4    | 5    | 6    |
|------------|------|------|------|------|------|------|------|------|------|
| error rate | 64.8 | 58.4 | 55.9 | 55.3 | 55.3 | 55.6 | 55.9 | 56.2 | 56.3 |

**c)** We performed a greedy search for optimal hyperparameters.

**d)** We use the same learning rate as the baseline.

Reviewer comment (Experiments):
What was the learning rate in the experiments — I assume it was taken from the baseline model. When adding your method, do you use the same learning rate as the baseline or also tune it separately?

Author response:
**W1.** There are many details missing about the hyper-parameters:

**a)** We applied PIF to TENT, EATA, PL, and SAR for each dataset, and each method has different loss functions, making it impossible to use the same hyperparameters across methods.

For ImageNet-C + EATA, an ablation study on the hyperparameter gamma yielded the following results:

| gamma      | 10    | 20    | 30    | 40    | 50    | 60    | 70    | 80    | 90    | 100   |
|------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
| error_rate | 51.74 | 51.51 | 51.38 | 51.35 | 51.37 | 51.43 | 51.51 | 51.62 | 51.81 | 52.03 |

SAR requires a larger gamma due to applying PIF only in the first update during backpropagation.

**b)** Beta values were chosen based on experimentation. For TENT+PIF on ImageNet-C, fixing gamma at 150 and varying beta produced the following error rates:

| beta       | 0    | 0.5  | 1    | 1.5  | 2    | 3    | 4    | 5    | 6    |
|------------|------|------|------|------|------|------|------|------|------|
| error rate | 64.8 | 58.4 | 55.9 | 55.3 | 55.3 | 55.6 | 55.9 | 56.2 | 56.3 |

**c)** We performed a greedy search for optimal hyperparameters.

**d)** We use the same learning rate as the baseline.

Reviewer comment (Experiments):
How is the model-trusting data defined? I assume you have used some entropy or maximum confidence threshold, but no details are given in the paper. Also, how sensitive is the method to this choice?

Author response:
**W2.** How is the model-trusting data defined?

**A.** Model-trusting data refers to samples with low entropy, similar to reliable samples in other methods. Adapting the model with data that gives near one-hot predictions is crucial.

Reviewer comment (Experiments):
The method is evaluated only on methods that update Batch Norms. Can we apply it to some other methods that update the whole model? For example COTTA, AdaContrast, Robust Mean Teacher?

Author response:
**W3.** Can the method be applied to other methods updating the whole model?

**A*.* Yes, but frequent updates to the last layer parameters would reduce memory and time efficiency.

Reviewer comment (Experiments):
The gamma parameter currently uses decay technique which requires setting a maximum number of iterations. This is a serious problem for TTA as we cannot know the maximum number of iterations. Can authors elaborate on that? Ideally, the authors would use a technique here that does not require maximum number of iterations.

Author response:
**W4.** The gamma parameter currently uses decay-technique which requires setting a maximum number of iterations.

**A.** We used a maximum number of iterations for decay techniques, but for cases like CIFAR-10, good performance was observed without decay. Additionally, for ImageNet-C, TENT+PIF showed a 1.4% improvement without decay, and PL showed a 2.7% improvement without decay.

Reviewer comment (Reproducibility):
There is no source code provided in the supplement which reduces the potential impact. It would be great to provide some version of the code for the rebuttal.

Author response:
**W5.** There is no source-code provided in the supplement.

**A.** The source code will be made public when the paper is published.

Reviewer comment (Experiments):
In the contributions the authors mention: "PIF regularizer which satisfies the limited access and low-resource constraints of TTA" — however, the low-resources constraint is never analyzed or discussed in depth later on.

Author response:
**W6.** The low-resources constraint is never analyzed or discussed in depth later on.

**A.** To utilize the influence function directly, heavy computations like Hessian inverse and full model gradient calculations are needed. We propose various approximations to calculate the influence function values through forward propagation only.

Reviewer comment (Experiments):
Batch size of 128 was used in the experiments. What would happen if we used some smaller batch size, for example 16? Are the gains of the proposed method more visible at large batches?

Author response:
**Q1.** What would happen if a smaller batch size, for example, 16, was used?

**A.** For TENT+PIF on CIFAR-10-C, the error rate decreased from 22.7% to 21.7% with a batch size of 16, showing a slight performance improvement compared to a batch size of 128. The reduced performance improvement is attributed to increased noise in the gradient with smaller batch sizes.

Reviewer comment (Experiments):
In Fig. 1, do you need some extra normalization for the model's weights?

Author response:
**Q3.** Do you need extra normalization for the model's weights in Fig. 1?

**A.** In Fig. 1, the model weights and test data features were normalized for plotting.

Reviewer comment (Experiments):
How do the changes in the feature extractor (even as small as changing the BN layers parameters) influence the calculation of PIF? How does the method work for longer sequences?

Author response:
**Q4.** How do changes in the feature extractor or longer sequences influence PIF calculation?

**A.** Changes in the feature extractor affect the input values of the last layer, influencing the calculation of PIF. PIF aligns the last layer gradient of the prototype with the last layer gradient of the test data, and its effectiveness is independent of sequence length.

Reviewer comment (Experiments):
Why didn't the authors compare SAR with LayerNorm and GroupNorm? As shown in SAR, SAR works much better using LN and GN. If PIF cannot be used in LN and GN and also cannot beat the results, can you convince me why I need PIF? Or can the author figure out how PIF can be used in LN and GN cases?

Author response:
SAR is a method designed for dynamic settings. Furthermore, the best performance was achieved when applying PIF to EATA.

Reviewer comment (Experiments):
If I understand the paper right, PIF is basically aligning test samples with prototypes in gradient space. In such a case, one natural question is have you tried aligning them in embedding space? How does it perform? If I can align the first-order derivative, can I align the second-order? Will it provide a better performance?

Author response:
As you correctly mentioned, PIF aligns prototypes and test data in the gradient space. Since we use prototypes as weights in the last layer, the alignment of prototypes and test data in the embedding space is already handled by existing methods such as EM loss or PL loss. Aligning them in the Hessian space is not feasible in TTA settings due to computational efficiency constraints.

Reviewer comment (Theory):
Since there is no label for test dataset, the test loss is often utiilized as entropy. Although I agree on that it is difficult to suggest new type of loss, I still think we cannot prove the learning direction is correct when trained with this type of loss. It could mean the influence calculated from the possibly wrong loss is also wrong.

Author response:
It is true that PIF depends on the form of the loss. I agree on the uncertainty of EM loss. We have derived PIF loss only for conventionally used EM and CE losses, but if there is a better loss, we can derive the corresponding PIF loss for it.

Reviewer comment (Theory):
I understood the weight vector of last layer may serve as a prototype of each class when there is no domain shift. However, if there is domain shift, can it also work as a prototype as the feature of source and features of target are distributed differently (more than Figure 1)? Or can we quantify the difference?

Author response:
We defined the prototype for model-trusting data. Model trusting data is defined as virtual data producing near one-hot predictions. To achieve near one-hot predictions, the features of the data should align well with the last layer weights. Therefore, we define the prototype of model-trusting data as the last layer weight. Thus, the definition of the prototype itself is not related to the shift between source and target distributions.

Reviewer comment (Reproducibility):
Can authors explain the scheme of random projection (which means approximations of the hessian to the identity) in more details? I think this trick is quite important for your method to reduce the computation complexity, yet it is not explained enough in the paper.

Author response:
he Hessian inverse is a factor that determines how much weighting to give between two gradients. However, due to the nature of PIF, which requires calculating IF during training, it is infeasible to compute the exact Hessian inverse. Therefore, we used the random projection, an approximation technique used in conventional IF, to set the Hessian to identity and perform calculations.

Reviewer comment (Experiments):
Why authors utilize L_PIF as regularizer? What happen if we do not use L_TTA? Why it would fail?

Author response:
The PIF derivation process itself regularizes to ensure positive impact on the model-trusting data when adapting test data with the original loss. Therefore, it is used as a regularizer. When using only PIF loss, it was effective, especially in small datasets like CIFAR-10-C, but had limitations for larger datasets like ImageNet-C.

Reviewer comment (Experiments):
The paper uses two approximation strategies for the influence function: parameter restriction and random projections, following previous work. Since this is the first application of the influence function in TTA, it is important to provide empirical evidence of the validity and impact of each approximation technique in TTA. A comparison with the exact influence function values, possibly by evaluating similarity or TTA performance, would be informative.

Author response:
**A.** The influence function was initially developed as a metric to understand black-box models, applicable after completing model training. For accurate influence function calculation, the inverse Hessian for the entire model parameter is required. However, as we measure the influence of test data on prototype loss at each iteration and raise its value, exact influence function computation is infeasible.
 The inverse Hessian is typically used for weighting between two gradients. However, our use of random projection techniques results in the Hessian becoming an identity matrix. As a result, the scale has a more significant impact on the influence function value than the direction of the gradient. To address this, we normalize the two gradients when approximating the influence function. While the exact influence function values are not comparable due to the nature of PIF, comparing performance with and without normalization, as shown in Figure 3 of the paper, demonstrates the impact.

Reviewer comment (Evaluation):
The claim that normalization of gradients reduces error and improves stability needs stronger support. Direct comparisons of the influence function values would be more convincing.

Author response:
**A.** The influence function was initially developed as a metric to understand black-box models, applicable after completing model training. For accurate influence function calculation, the inverse Hessian for the entire model parameter is required. However, as we measure the influence of test data on prototype loss at each iteration and raise its value, exact influence function computation is infeasible.
 The inverse Hessian is typically used for weighting between two gradients. However, our use of random projection techniques results in the Hessian becoming an identity matrix. As a result, the scale has a more significant impact on the influence function value than the direction of the gradient. To address this, we normalize the two gradients when approximating the influence function. While the exact influence function values are not comparable due to the nature of PIF, comparing performance with and without normalization, as shown in Figure 3 of the paper, demonstrates the impact.

Reviewer comment (Evaluation):
While approximations are applied to the influence function, it would be beneficial to discuss the time and memory requirements. A comparison of wall clock time and memory usage between the baseline and the proposed method could provide insight into the computational requirements.

Author response:
**A.** Measuring time and memory usage for CIFAR-10 with Gaussian noise data, TENT required 12.50 seconds and 5250 MiB, while TENT+PIF required 20.30 seconds and 7660 MiB. Both time and memory increased, but the error rate decreased by 1.7%.

Reviewer comment (Experiments):
They only provide the ablation study on $\gamma$ for one setting of CIFAR-10-C. They mentioned that "sustained robust performance of the hyperparameter beyond a certain threshold". However, the optimal choice of $\gamma$ provided in Table 5 varies, and the performance gap is quite small for some settings. It would be better to provide the sensitivity experiments on the other settings and datasets.

Author response:
**A.** For ImageNet-C dataset with TENT+PIF, fixing gamma at 150 and varying beta produced the following error rates:
| beta       | 0    | 0.5  | 1    | 1.5  | 2    | 3    | 4    | 5    | 6    |
|------------|------|------|------|------|------|------|------|------|------|
| error rate | 64.8 | 58.4 | 55.9 | 55.3 | 55.3 | 55.6 | 55.9 | 56.2 | 56.3 |

For ImageNet-C dataset with EATA+PIF, fixing beta at 2 and varying gamma produced the following error rates:
| gamma      | 10    | 20    | 30    | 40    | 50    | 60    | 70    | 80    | 90    | 100   |
|------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
| error_rate | 51.74 | 51.51 | 51.38 | 51.35 | 51.37 | 51.43 | 51.51 | 51.62 | 51.81 | 52.03 |

Reviewer comment (Experiments):
They did not provide the sensitivity results for $\beta$. It would be great to provide them.

Author response:
**A.** For ImageNet-C dataset with TENT+PIF, fixing gamma at 150 and varying beta produced the following error rates:
| beta       | 0    | 0.5  | 1    | 1.5  | 2    | 3    | 4    | 5    | 6    |
|------------|------|------|------|------|------|------|------|------|------|
| error rate | 64.8 | 58.4 | 55.9 | 55.3 | 55.3 | 55.6 | 55.9 | 56.2 | 56.3 |

For ImageNet-C dataset with EATA+PIF, fixing beta at 2 and varying gamma produced the following error rates:
| gamma      | 10    | 20    | 30    | 40    | 50    | 60    | 70    | 80    | 90    | 100   |
|------------|-------|-------|-------|-------|-------|-------|-------|-------|-------|-------|
| error_rate | 51.74 | 51.51 | 51.38 | 51.35 | 51.37 | 51.43 | 51.51 | 51.62 | 51.81 | 52.03 |