Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper investigates the pitfalls of online domain adaptation, that the model tends to collapse with small batch sizes and class imbalances at test time. The authors also propose two techniques to mitigate the problems. 
strength_and_weaknesses: **Strengh**
The paper is easy to follow. Authors also include explanations/intuitions to certain phenomena and design choices.

**Weakness:** Follows are my concerns. More detail below.
- The proposed method somewhat lacks novelty.
- The empirical evaluation can be improved and strengthened.
clarity,_quality,_novelty_and_reproducibility: The proposed method does not strike me as too novel. Clarity is good.
summary_of_the_review: Below are my concerns, which I hope the authors will address:
**The two solutions proposed by the authors do not seem new to me**:
- The entropy selective term $S(x)$ looks a lot like the one from EATA. In fact, it is almost the same thing, just without the coefficient term. It is also worrying to me that the authors do not mention nor discuss this similarity.
- The sharpness-aware minimization technique is also not new. It is adopted directly from SAM from the supervised loss to the self-supervised loss. On a side note, the authors mentioned a problem with gradient selection/clipping is that the threshold needs careful tuning. However, I would also argue that the hyper-parameter $\rho$ for the sharpness-aware entropy minimization also needs careful tuning, since it should depend on the norm of the parameters of the source-pretrained model.

**The empirical evaluation seems incomprehensive**
- The authors only validate their method for the case of entropy minimization. To the best of my knowledge, there are more surrogate losses that are often used for ODA. It would be beneficial to test if the method also helps in these cases.
- I wonder why the authors do not test their method with BN models (e.g., original resnet50)? If it does not perform well in this case, I suggest adding some explanation/discussion regarding this.
- It would also be beneficial to conduct experiments on domain shift datasets such as VisDA.


===========================
Post rebuttal:

I would like to thank the authors for their detailed response and the effort they put into this rebuttal.

They addressed most of my concerns, but some points remained (detailed below). For this reason, I increase my score from 3 to 5, since I think some arguments still need to be sharpened in the final draft.

- In the comparison with EATA and SAM, the authors pointed out some differences between their proposed method and these existing works (mostly about the purposes of the work, not much about the methodologies). With such similarities in the methodologies of the methods, it is absolutely important to acknowledge and discuss these details in the paper. As far as I am aware, the authors did not include any new discussions regarding this in the revision. Note that it is okay to (borrow)/(take motivation from) existing works, but discussion about the similarities, differences and novelties are needed for completeness.

- The authors mentioned they do not apply their method to other test time adaptation method, such as TTT, TTT+,... since they are computationally expensive (require several forward/backward passes). I find this arguement not convincing. If some methods are more expensive but offer better performance, they are still worth some value (some people even prefer these methods more for the performance). As the paper is deadling with TTA in the wild, I think it is important to cover a wide range of settings, including the TTA method.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors propose a test-time adaptation (TTA) method to deal with distribution shifts between training and test data. Their method aims to deal with the unstable issue of online model updating in TTA. A sharpness-aware and reliable entropy minimization method is designed. They also remove partial noisy samples and encourage model weights to go to a flat minimum to further stabilize TTA.
strength_and_weaknesses: * Strengths:
    
    1. This work provides detailed investigation of unstable reasons and careful analysis of failure cases in TTA. According to their investigation and analysis, a sharpness-aware and reliable method is proposed to solve those issues.

    2. Their discussion about norm layer is interesting. They also find weaknesses of only using GN and LN and provide a good solution to deal with such issues. The analysis of test sample gradients and the corresponding removing method is quite novel and efficient.

    3. Their experiments and ablation studies demonstrate that their analysis and hypothesis is indeed true, and the method they proposed is effective to improve the performance of TTA on ImageNet-C.

* Weaknesses:

    1. Their analysis about the norm layer is mainly based on empirical evidences. It is good to support their study with comprehensive experiments. However, I still wonder is there any intuitive explanation about choosing GN and LN instead of BN? This could help readers to better understand the analysis.

    2. In Fig. 6, the authors find even with fine-tuned hyper-parameters, gradient clipping yield worse results than no clipping, which is a bit counter-intuitive. Is there any explanation about the failure of gradient clipping even with fine-tuning?
clarity,_quality,_novelty_and_reproducibility: This paper is clearly presented in high quality. Their method is quite novel. Necessary implementation details required to reproduce their results are provided.
summary_of_the_review: Please address concerns mentioned in weaknesses.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a new method named sharpness-aware and reliable entropy minimization (SAR) to stabilize wild TTA. Specifically, to mitigate the collapse of group norm or layer norm models, SAR first filters samples with high and noisy gradients according to entropy loss. Furthermore, SAR introduces a sharpness-aware learning scheme to ensure that model weights remain at a flat minimum during optimization. Extensive experiments demonstrate the importance of Wild TTA and the effectiveness of SAR.
strength_and_weaknesses: Strength:
1. The authors introduce wild online TTA settings. For example, the test data may have 1) mixture distribution variation, 2) mini-batch, 3) online imbalanced label distribution variation, which are realistic during application.

2. Extensive experiments are conducted to illustrate the limitations of Batch Norm in TTA and further observed that batch-independent norms (e.g., Group Norm and Layer Norm) will be more stable than Batch Norm. The experiments provide new insights for the TTA community.

3. The authors observe that models with Group Norm or Layer Norm tend to collapse, especially when the severity of the distribution shift is high. They then propose a sharpness-aware learning scheme to alleviate the collapse problem, which is technically reasonable and novel within the scope of TTA.

4. The authors propose a sharpness-aware and reliable entropy minimization method to uniformly address the instability of previous methods under three wild TTA settings. Extensive experiments on ImageNet-C demonstrate the effectiveness of SAR.

Weakness:
1. This paper focuses on dynamic wild-test time adaptation settings. I am curious about under what circumstances does the third situation arise? Can the author provide some examples?

2. In Tables 2 and 4, all considered methods (including the proposed method) achieve worse TTA performance than baselines on ResNet50-GN with damaged Defoc. That is, all SOTA TTA methods cannot fit out-of-distribution samples. Could the authors discuss more about this phenomenon?

3. In Section 4, the description is a bit difficult to understand. Since these figures are informative, it is best to indicate which subgraph is being described when describing.

4. The numbers in section 4 are somewhat difficult to understand directly and the font size is too small. It's best to optimize these numbers for readability.

5. In Table 2, were the results for different damages measured with the same sample sequence?



clarity,_quality,_novelty_and_reproducibility: This paper is easy to read, novel, and of high quality. I think it has a big chance of being reproduced.
summary_of_the_review: This paper is easy to read and novel but still has some problems. Please refer to the weakness.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper focuses on fully test-time adaptation in dynamic and "wild" settings, in which (i) data from different target domains might be mixed at test time, (ii) test samples might not come in batches, and (iii) test samples might not be independently sampled in the sense that class frequency p(y) differs across batched. The authors identify batch norm as being harmful under such a setting and identify networks using group or layer norm to be preferable. Moreover, the authors propose sharpness-aware and reliable entropy minimization (SAR), to prevent model collapse caused by samples with large gradient norms. SAR consists of using the sharpness aware minimizer (SAM), entropy-based filtering of test samples used for adaptation, and a heuristic model recovery scheme. An empirical study in the dynamic "wild" settings and the proposed approaches is conducted on domain shifts with common corruptions.

strength_and_weaknesses: Strengths:
 * the proposed setting of "wild" test-time adaptation is of high practical relevancy and it is considerably more challenging that the standard setting of batched, single-target domain, class-balanced TTA. The authors also motivate and illustrate this setting well (compare Figure 1)
 * the paper does a good job in empirically identifying problematic parts of existing TTA approaches in the "wild" setting, namely (i) batch-norm's implicit single-domain assumption and reliance on large batches and (ii) model collapse caused by large gradient norms. 
 * Pragmatic solutions to these issues are identified that are not inherently novel but shown to be very efficient in the "wild" setting: (i) using group/layer norm instead of batch-norm and (ii) using SAR (SAM + entropy-based filtering + model recovery) to prevent model collapse.
 * Overall, the paper is strong on the empirical side and provides an extensive set of experiments on the different parts of the "wild" setting and the different components of the proposed wild TTA procedure. 
 * It is laudable that runtimes of different TTA-procedures are compared in Table 1. I would recommend to use the same unit (seconds) for all methods though. One question: why has SAR nearly the same runtime as TENT? I would expect that the double backward pass causes it to be somewhat slower. 

Weaknesses:
 * empirical evaluation is limited to target domains with common corruptions (e.g. ImageNet-C); investigation of other types of domain shifts like ImageNet-R would provide additional insights in terms of the generality of the proposed procedures.
 * the paper's comparison of normalization layers has a strong confounding factor in the overall model architecture: while batch-norm and group-norm are studied as part of a ResNet-50, layer-norm is studied as part of an ViT. Thus, a statement like "GN models are more suitable for self-supervised TTT than LN models since TTT+VitBase-LN is sensitive to different sample orders and has large variances over different runs." is problematic since it might not be that differences are due to the normalization layer (but rather to the model architectures). It would be preferable to remove the confounding factor of model architecture in this part.
 * The ablation study of SAR in Table 5 is nice but should be extended to cover all components of the wild setting and not only imbalanced labels
 * the model recovery scheme seems to be very helpful in the ViT-LN setting. To which extent would it also benefit TENT and EATA? That is: to which extent are benefits of SAR over TENT/EATA due to model recovery and not due to SAR?
 * not really a weakness, but a natural question arising would be: how would a model trained with SAM on the source domain perform in the comparisons given that SAM was found to be helpful at test-time?

clarity,_quality,_novelty_and_reproducibility: Overall the paper produces the settings, approaches, and experiments clearly and it is easy to follow the paper. The main novelty of the paper lies in the "wild" setting; the proposed procedure SAR is composed of well known components and not highly novel but still a useful contribution for approached the novel "wild" setting of TTA. The quality of the empirical evaluation is good even though a few improvements/extensions could be made (see above).
Reproducibility is given as details of the methods and experiments have been discussed and source code will be released upon acceptance. Moreover, experiments have been conducted based upon publicly available model checkpoints.

Minor: 
 * it was unclear to me which parameters have been adapted: only the affine parameters of normalization layers or all network parameters?
summary_of_the_review: Overall, I think this paper is a solid contribution that deserves acceptance at ICLR. The main contribution for me is to study and extend TTA in a more realistic "wild" setting, which should increase practical relevancy of TTA. The methods themselves are not novel but their combination and fit to "wild" TTA are sufficiently novel.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a new learning scheme, namely SAL (Sharpness-Aware & Reliable entropy minimization), to tackle the problem of test time adaptation (TTA) under distribution shift. The problem itself is practically important and the paper presents a sound solution that combines entropy minimization and sharpness aware minimization to mitigate TTA instabilities. The final algorithm is computationally efficient and works on real data. A thorough empirical study (including ablations) with encouraging results (both in the paper and its supplementary appendix) show the effectiveness of the approach both in terms of its computational efficiency and learning performance. The submission received 4 reviews, 3 of which are on clear accept side. One reviewer was initially more pessimistic about the results, specially around its novelty, but after multiple rounds of question/response between authors and the reviewer, the reviewer increased their score. In concordance with the reviewers, I think this submission provides a clear contribution to TTA and I recommend accept.
justification_for_why_not_lower_score: There is no reason for lower... the paper received great scores, authors provided a thorough response, and the proposed method is simple and working.
Decision: Accept: notable-top-5%