Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper presents a method for test time adaptation based on several techniques. These include a self-supervised adaptation objective based on log likelihood ratios, an additional regularizing objective to encourage diverse predictions, and an input transformation module that is also trained with the aforementioned objectives. Together, these techniques lead to better performance on ImageNet-C and ImageNet-R compared to Tent, a recent and similar test time adaptation method based on entropy minimization.
main_review: Originality
---
The paper itself notes that most of the techniques constituting the proposed method (diverse prediction regularization, input transformation module) are not novel. The self-supervised log likelihood ratio objective appears novel, as far as I am aware. I do not view the lack of originality as a substantial mark against this paper, as it is empirically focused and simply focuses on devising a better method for test time adaptation. And certainly, for this problem setting, the combination of the aforementioned techniques is novel and does lead to stronger empirical results than what has been previously reported.


Quality
---
The paper is of varying quality, and some improvements should be made as to this regard.

First, the paper could do a better job at motivating the proposed log likelihood ratio objective. Currently, there are two threads of motivation: intuitive, in the "non-saturating gradient" property, and empirical, in the improved results. But in their current form, these seem insufficient. For the intuitive motivation, it is not immediately clear why non-saturating gradients are desirable. If a model is already very confident about its prediction on a certain point, why and how should it derive any self-supervised signal to adapt? The idea behind entropy minimization methods like Tent is that confidence correlates with accuracy, so increasing confidence is desirable. This idea, of course, has flaws, but certainly it is intuitively understandable. It is not clear what the idea is behind the log likelihood ratio objective, or the general idea that adaptation can and should be performed when the model is confident. Furthermore, the gradient plot in Fig 1 does not seem to align well with the intuitive explanation for two reasons: the gradient of the entropy (green) is still non-trivially large for fairly high confidence, e.g., 0.9, and the caption says that entropy has "maximum gradient amplitude for low-confidence" prediction when this seems clearly false, e.g., the gradient around 0.5 is close to 0?

As for the empirical motivation that the objective simply performs better, this is nice but cannot fully substitute for better exposition of the idea itself. It is also unclear how important the input transformation module is, because I believe the paper does not include the baseline of Tent with this module (if I understand correctly, Tent+ only adds the diversity regularization). If such a baseline were to work very well, then it would turn out that the input transformation module is actually the most important piece, rather than changing the adaptation objective. Without results from this baseline, this possibility cannot be ruled out.

A more minor point is that it would be nice to include results in the setting where the model predicts on a batch directly after it adapts on the batch, rather than the current "offline" setting where several epochs are permitted on the entire test set before any predictions are made. In some practical applications, this would simply be infeasible, and predictions would have to be made right away. The paper could follow the general "online" setting that Tent uses.


Clarity
---
The paper is generally well written and structured. The main concern regarding clarity is that of the motivation behind the adaptation objective. Another more minor concern is whether the claim that the input transformation module can "undo distribution shifts" is justified, considering the mixed results in Table 2. Lastly, the exposition could be improved in 3.2.1 regarding the diversity regularization in running average mode. It seems like p_t ( y ) in L172 needs a stop gradient operator on p_t-1 ( y ), such that the gradients of the loss L_div in L173 are computed only with respect to p_t^emp. Otherwise, the method would involve storing a long history of model activations, and this seems unwieldy (and unlikely). If I am correct, it would be good to clarify this in the paper.


Significance
---
The paper focuses on an important topic (distribution shift) and an interesting emerging methodology (test time adaptation) with a moderate set of good results to support the proposed approach. Significance could be further improved through additional testbeds. For example, SVHN to MNIST/MNIST-M/USPS would offer another direct point of comparison to Tent. Alternatively, prior work [1] has shown that batch normalization based adaptation is ineffective on other test sets such as ImageNet-A and ImageNet-v2, so positive results on such test sets would be of high interest. The paper would be greatly strengthened through these additional experiments along with addressing the quality concerns noted above.

[1] Schneider et al, "Improving robustness against common corruptions by covariate shift adaptation". NeurIPS 2020.

---

Edit after reading the author response
---
As noted in my response below, my empirical concerns are resolved and I am thus upgrading my recommendation. I believe that, with the additional experiments, this work is above the bar for publication.
limitations_and_societal_impact: Yes, the authors have done so adequately, though additional discussion would also be nice, even if just in the appendix. Note that this is a minor point and does not contribute to my overall score.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: Focuses on test-time adaptation to distribution shifts, and proposes: i) a likelihood-ratio based loss with non-saturating gradients, ii) a diversity regularizer based on batch-wise entropy maximization, and iii) a learned input transformation module to undo the effect of distribution shift. Results are presented on standard test-time adaptation benchmarks.
main_review: Originality

– This work extends prior work (TENT) to optimize only batch norm parameters in addition to another “input transformation” module, to minimize a different loss function. 

– The originality of the stated technical contributions is somewhat limited:

i) As the authors acknowledge (L215), Yao et al. [40] were the first to propose the negative log likelihood-ratio loss 

ii) the diversity regularizer has also been proposed in prior work [11, 12]. The authors claim that they extend this to multiple batches, but this has also been done in Li et al. [A], which regularizes diversity by maximizing entropy over a history of model predictions over batches stored in a queue. 

– The shortcomings of the entropy minimization loss have been studied in prior work in domain adaptation: eg. [B] which proposes a new “max squares loss” and [C] which applies a Charbonnier penalty to entropy minimization. This work i) misses citing these highly related works, ii) clarifying what the contributions over these existing works is iii) empirically showing how the proposed loss compares against these prior works.

[A] Li, Bo, et al. "Rethinking distributional matching based domain adaptation." arXiv preprint arXiv:2006.13352 (2020).

[B] Chen, Minghao, Hongyang Xue, and Deng Cai. "Domain adaptation for semantic segmentation with maximum squares loss." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019.

[C] Yang, Yanchao, and Stefano Soatto. "Fda: Fourier domain adaptation for semantic segmentation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.

Quality

– The improvements over TENT on test-time adaptation are convincing, and the authors do well to benchmark multiple architectures.

– The paper lacks a clear ablation study -- only ITM appears to be ablated. What is the effect of removing L_{DIV} / L_{SLR}? Or optimizing L_{DIV} only over the current batch rather than as a moving average? 

– It’s also not entirely convincing that ITM helps – Table 2 shows that SSIM (SLR) > SSIM (SLR+IT) in 7/15 settings (though admittedly by narrower margins as the authors mention). In my opinion, this is inadequate evidence to claim that “ITM partially undoes the distribution shift”. The authors say that “illustrations of the effect of IT” is included in appendix, but I was unable to find this experiment.

Clarity

– The design of input transformation module is not clearly motivated. It appears to be a channel-wise learned affine transformation applied to a convex combination of the input and its (nonlinear) transformed version – while the first part seems identical to a batch-normalization layer, what is the motivation/intuition behind the second part? Have the authors experimented with alternatives?

– L162 states “assuming knowledge of the class distribution .. on the target domain”, but does not clarify how this is estimated / approximated. Lacking access to labels on the target, how is p_{D’}(y) computed? Or is it assumed to be the uniform distribution?

– Minor: Why is there a separate “pseudo labels” row in Table 1? Is it not identical to the previous “no adaptation” row by definition? 

– In Figure 2, HLR outperforms SLR on ImageNet-C and clean ImageNet, but the opposite happens on ImageNet-R. Have the authors 
investigated the reason for this? When is it more appropriate to use one over the other?

– Minor: Why is SHOT not directly included as a point of comparison in say Table 1, but rather in the appendix?

Significance

– The paper studies an interesting and practical setting of test-time adaptation, and the proposed approach appears to improve upon the existing state-of-the-art (TENT) on standard benchmarks.

Overall

The paper studies the important and practical setting of test-time adaptation, and extends prior work (TENT) via a new loss function with non-saturating gradients and a new input transformation module, which lead to improved performance. The paper is mostly well-written and easy to follow. I have concerns primarily about the originality of the paper’s stated contributions (primarily, not citing / comparing to highly relevant work in DA studying the shortcomings of entropy minimization) and the quality of its experimental section (primarily ablations), and so believe that in its current form it is below the bar for acceptance.


---post-rebuttal---

I have read the author response and other reviews. While I still believe the technical novelty is slightly limited, I think the paper is well-written and the rebuttal addresses most of my primary concerns, and recommend marginal acceptance.
limitations_and_societal_impact: Authors adequately addressed the limitations and potential negative societal impact.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: Fully test-time adaptation is a promising approach to handling data shift and making the DNNs more robust. This paper presents a new method for fully test-time adaptation. Specifically, they propose a non-saturating surrogate of entropy and combine it with the diversity regularizer. They empirically demonstrate the efficacy of the proposed method, comparing it against a recently proposed test-time adaptation method, Tent, using various backbone networks. 
main_review: ---
Overall, the paper is very clearly written, and it was a fan to read. The proposed method might not be a revolutionary one, a combination of several (known) techniques, yet it is well described, and the experimental results are extensive enough to show benefits. They also provide results on various backbone networks, which is informative for future researches. However, several details regarding the motivation and implementation are unclear for me, making me hesitate to vote for full acceptance of the paper. 

---
**Comments/Questions**

1. The motivation behind the proposal of a non-saturating objective is that information about more confident samples should be affected more on test-time adaptation. However, I do not fully agree with this point, as it does not affect the training just because the model parameters are already suitable for the data point. 
2. If I correctly understand the paper, there are two main differences between the Tent and the proposed method: (1) loss function and (2) tuned parameter. However, it is not clear to me why do we need to change the latter one. Can you discuss more that? Specifically, what is the potential problem of transforming only the parameter of BN as with the Tent?
3. Figure 2 suggests that the proposed method often negatively affects the performance on the clean dataset, compared to the Tent and Tent+. Why does it happen? Besides, it is not clear to me how exactly the performance on the clean dataset is measured. More specifically, is the model adapted on the clean dataset or adapted on the corrupted dataset? 
4. In section 4, the authors say that the experiment uses cosine decay scheduling. However, looking at the code, this doesn't seem to apply to Tent.  It seems that the comparison is unfair due to this. 
5. Learning five epochs during the test seems a bit strange to me. This is because the deployed model usually needs to make correct predictions at that moment; there is no point in going back in time and correcting the predictions. 
limitations_and_societal_impact: The paper mentions a little about the difficulty of using the method in safety-critical downstream tasks and describes the diversity regularizer possibly compensates the issue. However, it is unclear to me how the proposed regularizer help to migrate the issue. I feel this is the most critical issue of fully test-time adaptation, and it is worth discussing more. 

Besides that, I can not find any statements about limitations. For example, it can not directly be used for the regression, as the non-saturating modification assumes the discrete prediction (although Tent has the same issue). Would you please discuss more about such a limitation of this work?
review_title: null

Meta-review: This paper presents a method for improving DNNs performance on distribution shifts via test time adaptation based on several techniques (a non-saturating loss function, diversity maximization and input transformation module). The paper requires significant modifications to be ready for publication in terms of clarity, additional comparison experiments, discussing the effectiveness of the method in more details, etc (see reviews and discussions for details). The discussions have not been enough to convince me and reviewers to accept the paper as is. I encourage authors to update the paper in light of discussions and feedback and submit to a future venue.
Decision: Reject