Towards Stable Test-time Adaptation in Dynamic Wild World
Authors: Shuaicheng Niu
Jiaxiang Wu
Yifan Zhang
Zhiquan Wen
Yaofo Chen
Peilin Zhao
Mingkui Tan
Venue: ICLR 2023 notable top 5% (2023)
Keywords: Test-time adaptation
Roustness

Abstract: Test-time adaptation (TTA) has shown to be effective at tackling distribution shifts between training and testing data by adapting a given model on test samples. However, the online model updating of TTA may be unstable and this is often a key obstacle preventing existing TTA methods from being deployed in the real world. Specifically, TTA may fail to improve or even harm the model performance when test data have: 1) mixed distribution shifts, 2) small batch sizes, and 3) online imbalanced label distribution shifts, which are quite common in practice. In this paper, we investigate the unstable reasons and find that the batch norm layer is a crucial factor hindering TTA stability. Conversely, TTA can perform more stably with batch-agnostic norm layers, i.e., group or layer norm. However, we observe that TTA with group and layer norms does not always succeed and still suffers many failure cases. By digging into the failure cases, we find that certain noisy test samples with large gradients may disturb the model adaption and result in collapsed trivial solutions, i.e., assigning the same class label for all samples. To address the above collapse issue, we propose a sharpness-aware and reliable entropy minimization method, called SAR, for further stabilizing TTA from two aspects: 1) remove partial noisy samples with large gradients, 2) encourage model weights to go to a flat minimum so that the model is robust to the remaining noisy samples. Promising results demonstrate that SAR performs more stably than prior methods and is computationally efficient under the above wild test scenarios. 

Published as a conference paper at ICLR 2023
TOWARDS STABLE TEST-TIME ADAPTATION IN DYNAMIC WILD WORLD
Shuaicheng Niu 14: Jiaxiang Wu²*, Yifan Zhang Zhiquan Wen', Yaofo Chen', Peilin Zhao², Mingkui Tan
senscêma 1., scut. edu. cn; m mingkuitan@scus edu. en
South China University of Technology' Tencent AI Lab? National University of Singapore Key Laboratory of Big Data and Intelligent Robot, Ministry of Education Pazhou Laboratory
ABSTRACT
Test-time adaptation (TTA) has shown to be effective at tackling distribution shifts between training and testing data by adapting a given model on test samples. How- ever, the online model updating of TTA may be unstable and this is often key obstacle preventing existing TTA methods from being deployed in the real world. Specifically. TTA may fail to improve or even harm the model performance when test data have: 1) mixed distribution shifts, 2) small batch sizes, and 3) online imbalanced label distribution shifts, which are quite common in practice. In this paper, we investigate the unstable reasons and find that the batch norm layer a crucial factor hindering TTA stability. Conversely, TTA can perform more stably with batch-agnostic norm layers, i.c., group or layer norm. However, we observe that TTA with group and layer norms does not always succeed and still suffers many failure cases. By digging into the failure cases, we find that certain noisy test samples with large gradients may disturb the model adaption and result in collapsed trivial solutions, i.e. assigning the same class label for all samples. To address the above collapse issue, we propose sharpness-aware and reliable en- tropy minimization method, called SAR, for further stabilizing TTA from two aspects: 1) remove partial noisy samples with large gradients, 2) encourage model weights to go to a flat minimum so that the model is robust to the remaining noisy samples. Promising results demonstrate that SAR performs more stably over prior methods and is computationally efficient under the above wild test scenarios. The source code is available at https //tp//gtttt///////////////tttttt com/ mr 1t/mmmrreegeege weeeeegggllannnneeeeeeeeeee /SAR.
INTRODUCTION
Deep neural networks achieve excellent performance when training and testing domains follow the same distribution (He et al., 2016; Wang et al., 2018; Choi et al., 2018). However, when domain shifts exist, deep networks often struggle to generalize. Such domain shifts usually occur in real ap- plications, since test data may unavoidably encounter natural variations corruptions (Hendrycks & Dietterich, 2019; Koh et al., 2021), such as the weather changes (e.g.. snow, frost, fog), sensor degradation (e.g.. Gaussian noise, defocus blur), and many other reasons. Unfortunately, deep mod- els can be sensitive to the above shifts and suffer from severe performance degradation even if the shift is mild (Recht et al., 2018). However, deploying a deep model on test domains with distribution shifts is still an urgent demand, and model adaptation is needed in these cases.
Recently, numerous test-time adaptation (TTA) methods (Sun et al., 2020; Wang et al., 2021; Iwa- sawa & Matsuo, 2021; Bartler et al., 2022) have been proposed to conquer the above domain shifts by online updating a model on the test data, which include two main categories, i.e., Test-Timo Training (TTT) (Sun et al., 2020; Liu et al., 2021) and Fully TTA (Wang al., 2021; Niu et al., 2022a). In this work. we focus on Fully TTA since it is more generally to be used than TTT two aspects: it does not alter training and can adapt arbitrary pre-trained models to the test data without access to original training data; ii) it may rely on fewer backward passes (only one or less than one) for each test sample than TTT (see efficiency comparisons of TTT, Tent and EATA in Table 6).
"quual contribution. 'Corresponding author. Work done when S. Niu works intern in Tencent AT Lab.

---

Published as a conference paper at ICLR 2023
0/0 i Sample from domain 1/2/3
Tant acouragy
Teat samplis
Tent acouray
0/0/A Sample from ciass 1/2/5
Tant socuracy
Single domain:
Batch samp es:
Randomly shuffled label distribution:
3
TTA
Prior
600
6%
000
UBG
DOG
28 7%
Mixnd domains:
Single sample:
Cnlinn imballanced labal distribution shift
2 Wild 1TA
2.5%
0.1%
1.9%
c-
(6) TTA under mixed domain shifts
(b) TTA with single sample
(c) TTA under online imbalanced label shift
Figure 1: An illustration practical/wild test-time adaptation (TTA) scenarios, in which prior online TTA methods may degrade severely. The accuracy of Tent (Wang et al., 2021) is measured on ImageNet-C of level with ResNe150-BN (15 mixed corruptions in (a) and Gaussian in (b-c)).
TTA has been shown boost model robustness to domain shifts significantly. However, its excellent performance is often obtained under some mild test settings, e.g., adapting with a batch of test samples that have the same distribution shift type and randomly shuffled label distribution (see Figure 1 ①). In the complex real world, test data may come arbitrarily. As shown in Figure 1 @, the test scenario may meet: i) mixture of multiple distribution shifts, ii) small test batch sizes (even single sample), iii) the ground-truth test label distribution Qr(y) is online shifted and Qr(y) may be imbalanced at each time-step t. In these wild test settings, online updating a model by existing TTA methods may be unstable, i.e., failing to help even harming the model's robustness.
To stabilize wild TTA, one immediate solution is to recover the model weights after each time adaptation of a sample mini-batch, such as MEMO (Zhang al., 2022) and episodic Tent (Wang et al., 2021). Meanwhile, DDA (Gao et al., 2022) provides potentially effective idea to address this issue: rather than model adaptation, seeks to transfer test samples the source training distribution (via a trained diffusion model (Dhariwal & Nichol, 2021)), in which all model weights are frozen during testing. However, these methods cannot cumulatively exploit the knowledge of previous test samples to boost adaptation performance, and thus obtain limited results when there are lots of test samples. In addition, the diffusion model in DDA is expected to have good generalization ability and can project any possible target shifts to the source data. Nevertheless, this is hard to be satisfied as far as goes, e.g., DDA performs well on noise shifts while less competitive on blur and weather (see Table 2). Thus, how to stabilize online TTA under wild test settings is still an open question.
In this paper, we first point out that the batch norm (BN) layer (loffe & Szegedy, 2015) is a key obstacle since under the above wild scenarios the mean and variance estimation in BN layers will be biased. In light of this, we further investigate the effects of norm layers in TTA (see Section 4) and find that tre-trained models with batch-agnostic norm layers (i.e., group norm (GN) (Wu & He, 2018) and layer norm (LN) (Ba et al., 2016)) are more beneficial for stable TTA. However, TTA on GN/LN models does not always succeed and still has many failure cases. Specifically, GN/LN mod- els optimized by online entropy minimization (Wang et al., 2021) tend to occur collapse, i.e., pre- dicting all samples to a single class (see Figure 2), especially when the distribution shift severe. To address this issue, we propose a sharpness aware and reliable entropy minimization method (namely SAR). Specifically, we find that indeed some noisy samples that produce gradients with large norms harm the adaptation and thus result in model collapse. To avoid this, we filter partial samples with large and noisy gradients out of adaptation according to their entropy. For the remaining samples, we introduce a sharpness-aware learning scheme to ensure that the model weights are optimized to a flat minimum, thereby being robust to the large and noisy gradients/updates.
norm layers (i.e,, GN and LN) are more beneficial than BN to stable test-time adaptation under wild test settings, i.e., mix domain shifts, small test batch sizes and online imbalanced label distribution shifts (see Figure 1). (2) We further address the model collapse issue of test-time entropy minimization on GN/LN models by proposing sharpness-aware and reliable (SAR) optimization scheme, which jointly minimizes the entropy and the sharpness of entropy of those reliable test samples. SAR is simple yet effective and enables online test-time adaptation stabilized under wild test settings.
Main Findings and Contributions. (1) We analyze and empirically verify that
2 PRELIMINARIES
We revisit two main categories of test-time adaptation methods in this section for the convenience of further analyses, and put detailed related work discussions into Appendix A due to page limits.
2

---

Published as a conference paper at ICLR 2023
) with parameter + where X; E &rain (the training data space) and yi E C (the label space). The goal of test-time adaptation (Sun et al., 2020; Wang et al., 2021) is to boost fx(x) on out-of-distributio test samples Drest = {x; ==1-- where Xj E &eest (testing data space) and Nest * Xtraia- Sun et al. (2020) first propose the TTT pipeline, in which at training phase model is trained source Dirain via both cross-entropy Loe and self-supervised rotation prediction (Gidaris et al., 2018) Lst
Test-time Training (TTT). Let fe(x) denote model trained on Dtrain {xx,, V/1 1:))),[,
min ExEDiamn Ob Q.) + [x Ob, θ,)] O,
(1)
where O is the task-shared parameters (shadow layers), O, and O are task-specific parameters (deep layers) for LCE and Ls. respectively. At testing phase, given a test sample x, TTT first updates the model with self-supervised task: 0, < arg mine, (x; Qb, O2) and then use the updated model weights 6 to perform final prediction via f(x: 0/ 0.).
Fully Test-time Adaptation (TTA). The pipeline of TTT needs to alter the original model training process, which may be infeasible when training data are unavailable due to privacy/storage concerns. To avoid this, Wang et al. (2021) propose fully TTA, which adapts arbitrary pre-trained models for a given test batch by conducting entropy minimization (Tent): min E. Dn log 9. where (c=x)((())) and e denotes class e. This method is more efficient than TTT as shown in Table 6.
3 STABLE ADAPTATION BY TEST ENTROPY AND SHARPNESS MINIMIZATION
Test-time Adaptation (TTA) in Dynamic Wild World. Although prior TTA methods have exhib- ited great potential for out- of-distribution generalization, its success may rely on some underlying test prerequisites (as illustrated in Figure 1): 1) test samples have the same distribution shift type; 2) adapting with a batch of samples each time, 3) the test label distribution is uniform during the whole online adaptation process, which, however, are easy to be violated in the wild world. wild scenarios (Figure 1 (),, prior methods may perform poorly or even fail. In this section, we seek to analyze the underlying reasons why TTA fails under wild testing scenarios described in Figure from unified perspective (c.f. Section 3.1) and then propose associated solutions (c.f. Section 3.2).
3.1 WHAT CAUSES UNSTABLE TEST-TIME ADAPTATION?
We first analyze why wild TTA fails by investigating the norm layer effects in TTA and then dig into the unstable reasons for entropy-based methods with batch-agnostic norms, e.g., group norm.
Batch Normalization Hinders Stable TTA. In TTA, prior methods often conduct adaptation on pre-trained models with batch normalization (BN) layers (loffe & Szegedy, 2015), and most of them are built upon BN statistics adaptation (Schneider et al., 2020; Nado et al., 2020; Khurana al., 2021; Wang et al., 2021; Niu et al., 2022a; Hu et al., 2021; Zhang et al., 2022). Specifically, for a layer with d-dimensional input (z<11) ......))))
). the batch normalized output are: ((()
(k) + 3(k), where (k) = (z) E 1// Var [z=(k]] Here, (1k) and g(k)) are lcarnable affine parameters. BN adaptation methods calculate mean E[],(*) and variance Var[z:(*) over (a batch of) test samples. However, in wild TTA, all three practical adaptation settings (in Figure 1) in which TTA may fail will result in problematic mean and variance estimation. First, BN statistics indeed represent distribution and ideally each distribution should have its own statistics. Simply estimating shared BN statistics of multiple distributions from mini batch test samples unavoidably obtains limited performance, such as in multi tailiiiassk/dmmmnn learning (Wu & Johnson, 2021). Second, the quality of estimated statistics relies on the batch size, and it is hard to use very few samples (i.e., small batch size) to estimate it accurately. Third, the imbalanced label shift will also result in biased BN statistics towards some specific classes in the dataset. Based the above, we posit that batch-agnostic norm layers, i.e., agnostic to the way samples are grouped into a batch, are more suitable for performing TTA, such as group norm (GN) (Wu & He, 2018) and layer norm (LN) (Ba et al., 2016). We devise our method based on GN/LN models in Section 3.2.
To verify the above claim, we empirically investigate the effects of different normalization layers (including BN, GN, and LN) in TTA (including TTT and Tent) in Section 4. From the results, observe that models equipped with GN and LN are more stable than models with BN when
3

---

Published as a conference paper at ICLR 2023
16) Model collapse on level
(b) No collapse on level
(c) Evolution gradients norm
(1) Entropy gradients nom
Predicted Cless
1000
750
Prediced Dasss
1000
750
Gradlene (Avg.)
40
Severity lovel
Severity leve 3
Entrany of Sample
Aree
start collapue
500
500
20
start collapse
230
250
Area
1
0.0
250
500 Online Batch
750
0
250
500
750
0
250
500 Online Batch
750
100
200
300
Online Batch
Gradient Norm of Sample
Figure 2: Failure case analyses (a-c) of online test-time entropy minimization (Wang et al., 2021). (a) and (b) record the model predictions during online adaptation. (c) illustrates how gradients norm evolves with and without model collapse. (d) investigates the relationship between the sample's s entropy and gradients norm. All experiments are conducted on shuffled ImageNet-C of Gaussian noise with ResNet50 (GN), and larger (severity) level denotes more severe distribution shift.
performing online test-time adaptation under three practical test settings (in Figure 1) and have fewer failure cases. The detailed empirical studies are put in Section 4 for the coherence of presentation.
Online Entropy Minimization Tends to Result in Collapsed Trivial Solutions, i.e., Predict All Samples to the Same Class. Although TTA performs more stable on GN and LN models, it does not always succeed and still faces several failure cases (as shown in Section 4). For example, en- tropy minimization (Tent) on GN models (ResNet50-GN) tends to collapse, especially when the distribution shift extent is severe. In this paper, we aim to stabilize online fully TTA under various practical test settings. To this end, we first analyze the failure reasons, in which we find models are often optimized to collapse trivial solutions. We illustrate this issue in the following.
During the online adaptation process, we record the predicted class and the gradients norm (pro- duced by entropy loss) of ResNet50-GN on shuffled ImageNet-C of Gaussian noise. By comparing Figures 2 (a) and (b), entropy minimization is shown to be unstable and may occur collapse when the distribution shift is severe (i.e, severity level 5). From Figure 2 (a), as the adaptation goes by, the model tends to predict all input samples to the same class, even though these samples have dif- ferent ground-truth classes, called model collapse. Meanwhile, we notice that along with the model starts to collapse the é 2--norm of gradients of all trainable parameters suddenly increases and then degrades to almost 0 (as shown in Figure 2 (c)), while on severity level 3 the model works well and the gradients norm keep in a stable range all the time. This indicates that some test samples produce large gradients that may hurt the adaptation and lead to model collapse.
3.2 SHARPNESS-AAAN AND RELIABLE TEST-TIME ENTROPY MINIMIZATION
Based on the above analyses, two most straightforward solutions to avoid model collapse are filtering test samples according to the sample gradients or performing gradients clipping. However, these are not very feasible since the gradients norms for different models and distribution shift types have different scales, and thus it is hard to devise a general method set the threshold for sample filtering or gradient clipping (see Section 5.2 for more analyses). We propose our solutions as follows.
Reliable Entropy Minimization. Since directly filtering samples with gradients norm is infeasible, we first investigate the relation between entropy loss and gradients norm and seek to remove samples with large gradients based on their entropy. Here, the entropy depends on the model's 's output class number C' and it belongs to (0. In C") for different models and data. In this sense, the threshold for filtering samples with entropy is easier to select. As shown Figure 2 (d), selecting samples with small loss values can remove part of samples that have large gradients (a (area@ @ 1) out of adaptation. Formally, let (x; θ) be the entropy of sample x, the selective entropy minimization is defined by:
min S(x)E(x; θ), where S(x) 2 ([E(x=))=,)(x)) 8
Here, θ denote model parameters, (-) is indicator function and Eg is a pre-defined parameter. Note that the above criteria will also remove samples within area 2 in Figure 2 (d), in which the samples have low confidence and thus are unreliable (Niu et al., 2022a).
Sharpness-aware Entropy Minimization. Through Eqn. (2), we have removed test samples in area 1&2 in Figure 2 (d) from adaptation. Ideally, we expect to optimize the model via samples only in area@3 3, since samples in area@ 4 still have large gradients and may harm the adaptation. However, it is hard to further remove the samples in rea @ via filtering scheme. Alternatively,
4

---

Published as a conference paper at ICLR 2023
(a) TTT, Severity Level
(b) Tent, Severity Level
(c) TTT. Severty Level 3
(d) Tent, Severty Level
Accuracy (%)
40
40
40
20
20
800-88111 1EA1
20
20
0
64
Batch Size
Batch Size
Batch Size
Batch Size
Figure 3: Batch size effects of different TTA methods under different models (different normaliza- tion layers). Experiments are conducted on ImageNet-C of Gaussian noise. We report mean and standard deviation of 3 runs with different random seeds. 'na' denotes no adapt accuracy. Note that except for Vit-LN, the standard deviation is too small to display in the figures.
we seek to make the model insensitive the large gradients contributed by samples in areaß 4. Here, we encourage the model to go to a flat area of the entropy loss surface. The reason is that a flat minimum has good generalization ability and is robust to noisy/large gradients, i.e., the noisy/large updates over the flat minimum would not significantly affect the original model loss, while a sharp minimum would. To this end, we we jointly minimize the entropy and the sharpness of entropy loss by:
min E (x; θ) where ESA(x: θ max Exx; θ + E). ||EllaSp
Here, the inner optimization seeks to find a weight perturbation in Euclidean ball with radius that maximizes the entropy. The sharpness is quantified by the maximal change of entropy be- tween Q and θ + e. This bi-level problem encourages the optimization to find flat minima. To address problem (3), we follow SAM (Foret et al., 2021) that first approximately solves the inner optimization via first-order Taylor expansion, iee.,
e' (@) - arg maxmmxE xx; θ + c) arg argmaxE(xx ①) + + VVVeE(x; θ) arg maxe T Ee O). ||cllaSp
I<la5p
Then, E(O) that solves this approximation is given by the solution to a classical dual norm problem:
E(O) = sign EV θ)) V / [(; 0)1l2
(4)
Substituting é(0) back into Eqn. (3) and differentiating, by omitting the second-order terms for computation acceleration, the final gradient approximation is:
VeESA = EeE (+2(0))
(5)
Overall Optimization, In summary, our sharpness-aware and reliable entropy minimization is: min Fi
where Sxx) and ESA(x: Q) are defined in Eqns. (2) and (3) respectively, Ô C θ denote learnable parameters during test-time adaptation. In addition, to avoid few extremely hard cases that Eqn. (6) may also fail, we further introduce Model Recovery Scheme. We record a moving average em of entropy loss values and reset Ô to be original once 6m is smaller than a small threshold eg. since models after occurring collapse will produce very small entropy loss. Here, the additional memory costs are negligible since we only optimize affine parameters in norm layers (see Appendix C.2 for more details). We summarize the details our method in Algorithm in Appendix B.
EMPIRICAL STUDIES OF NORMALIZATION LAYER EFFECTS IN TTA
This section designs experiments to illustrate how test-time adaptation (TTA) performs on models with different norm layers (including BN, GN and LN) under wild test settings described in Fig- ure 1. We verify two representative methods introduced in Section 2, i.c., self-supervised TTT (Sun et al., 2020) and unsupervised Tent (a fully TTA method) (Wang et al., 2021). Considering that the norm layers are often coupled with mainstream network architectures, we conduct adaptation on ResNet-50- BN (R-50-BN), ResNet-50-Gr (R-50-GN) and VitBase-LN (Vit-LN). All adopted model weights are public available and obtained from forchiision or t timm repository (Wight- man, 2019). Implementation details of experiments in this section can be found in Appendix C.2.
5

---

Published as a conference paper at ICLR 2023
(a) TTT, Severity Level 5
(b) Tent, Severity Level
(c) TTT, Severity Level 3
(d) Tent, Severity Level
60
60
Accuracy (%)
Aey aliae
Auy ideat
4LC
60-
60
FD
40-
20
20
40-
40
20
20-
R-50-BN
R-50-GN VitDase-UX
R:50 UN
R-50-GN VitBase-LN
R-50-BN
R-50-GN Vitaase
R:50 BN
8.50 GN VitB UN
Figure 4: Performance of TTA methods different models (different norm layers) under the mix-
ture of 15 different corruption types (ImageNet-C) We report mean& over 3 independent runs.
TTT, Severity Level
(b) Tent, Severity Level
TTT, Severity Level
Tent, Severity
Accuracy (%)
40
40
60
60
R-50-UN
R:50 GN
20
20
VIBess-LN
20
20
1 1k 2k
3K
4k 5k
1k 2k
3k
4K
5k
1k 2k
3k
4k 5k
1k 2k
3k
4k 5k
Imbalance Ratio
Imbalance Ratio
Imbalance Ratio
Imbalance Ratio
Figure 5: Performance of TTA methods with different models (different norm layers) under online imbalanced label distribution shifts on ImageNet-C (Gaussian noise). We report mean&stdev. re- sults of runs. Note that except for VitBase-LN,, the stdev. is too small to display in the figures.
(1) Norm Layer Effects TTA Under Small Test Batch Sizes. We evaluate TTA methods (TTT and Tent) with different batch sizes (BS), selected from {1, 2, 4. 8, 16, 32, 64). Due to GPU memory limits, we only report results of BS up to 8 or 16 for TTT (in original TTT BS is I), since TTT needs to augment each test sample multiple times (for which we set to 20 by following Niu et al. (2022a)).
From Figure 3, we have: i) For Tent, compared with R-50-BN, R-50-GN and Vit-LN are less sensi- tive to small test batch sizes. The adaptation performance of R-50-BN degrades severely when the batch size goes small (<8), while R-50-GN/Vit-LN show stable performance across various batch sizes (Vit-LN on levels 5&3 and R-50-GN level 3, in subfigures (b)& It is worth noting that Tent with R-50-GN and Vit-LN not always succeeds and also has failure cases, such as R-50-GN on level 5 (Tent performs worse than no adapt), which analyzed in Section 3.1. ii) For TTT, all R-50- BN/GN and Vit-LN can perform well under various batch sizes. However, TTT with Vit-LN is very unstable and has a large variance over different runs, showing that Vit+vitBase very sensitive to different sample orders. Here, TTT performs well with R-50-BN under batch size is mainly benefited from TTT applying multiple data augmentations to a single sample to form a mini-batch.
(2) Norm Layer Effects in TTA Under Mixed Distribution Shifts. We evaluate TTA methods on models with different norm layers when test data come from multiple shifted domains simulta- neously. We compare 'no adapt', 'avg. adapt' (the average accuracy of adapting each domain separately) and 'mix adapt' (adapting on mixed and shifted domains) accuracy on ImageNet-C con- sisting of 15 corruption types. The larger accuracy gap between 'mix adapt' and "avg. adapt' indicates the more sensitive to mixed distribution shifts.
From Figure 4, we have: i) For both Tent and TTT, R-50-GN and Vit-LN perform more stable than R-50-BN under mix domain shifts. Specifically, the mix adapt accuracy of R-50-BN is consistently poor than the average adapt accuracy across different severity levels (in all subfigures (a-d).. In contrast, R-50-GN and Vit-LN are able to achieve comparable accuracy of mix and average adapt, i.e., TTT on R-50-GN (levels 5&3) and Tent on Vit-LN (level 3). ii) For R-50-GN and Vit-LN, TTT performs more stable than Tent. To be specific, Tent gets 3/4 failure cases (R-50-GA on levels 5&3, Vit-LN on level 5), which is more than that of TTT. iii) The same as Section 4 (1), TTT on Vit-LN has large variances over multiple runs, showing TTT++Vtt-L is sensitive to different sample orders.
(3) Norm Layer Effeets in TTA Under Online Imbalanced Label Shifts. As in Figure/ 1 (c), during the online adaptation process, the label distribution Qv()) at different time-steps L may be different (online shift) and imbalanced. To evaluate this, we first simulate this imbalanced label distribution shift by adjusting the order of input samples (from test set) as follows.
Online Imbalanced Label Distribution Shift Simulation. Assuming that we have totally T time-steps and T equals to the class number C. We set the probability vector Qc(v) [y1, 92. 4c]. where qe 9max if t and 9e 9min & (1 4mammm) (C 1) if c t. Here, qmaz/ fmaz/9min 9mi denotes the imbalance ratio. Then, at each t E (IC), we sample M images from the test set according to (v). Based on ImageNet-C (Gaussian noise), we construct new testing set that has

---

Published as a conference paper at ICLR 2023
online imbalanced label distribution shifts with totally (M)( (T) images. Note that pre-shuffle the class orders in ImageNet-C, since we cannot know which class will come in practice.
From Figure 5, we have: i) For Tent, R-50-GN and Vit-LN are less sensitive than R-50-BN to online imbalanced label distribution shifts (see subtigures (b)&(d)). Specifically, the adaptation accuracy of R-50-BN (levels 5&3) degrades severely as the imbalance ratio increases. In contrast, R-50-GN and Vit-LN have the potential to perform stably under various imbalance ratios (e.g., R-50-GN and Vit-LN on level 3). ii) For TTT, all R-50-BN/GN and Vit-LN perform relatively stable under label shifts, except for VIT+V has large variances. The adaptation accuracy will also degrade but not very severe as the imbalance ratio increases. iii) Tent with GN is more sensitive to the extent of distribution shift than BN. Specifically, for imbalanced ratio 1 (all (y) are uniform) and severity level 5, Tent+R-50-C0 fails and performs poorer than no adapt, while Tent+R-50-BN works well.
(4) Overall Observations. Based on all the above results, we have: i) R-50-GN and Vit-LN are more stable than R-50-BN when performing TTA under wild test settings (see Figure 1). However, they do not always succeed and still suffer from several failure cases. ii) R-50-GN is more suitable for self-supervised TTT than Vit-LN, since TTT+Vit-LN is sensitive to different sample orders and has large variances over different runs iii) Vit-LN is more suitable for unsupervised Tent than R-50-GN, since Tent+R-50-GN is easily to collapse, especially when the distribution shift is severe.
5 COMPARISON WITH STATE-OF-THE-ARTS
Table 1: Efficiency compari-
son for processing 50,000 im- ages (Gaussian noise, level 5 ImageNet-C) via single V100 GPU on ResNe150-GN.
Dataset and Methods. We conduct experiments based on ImageNet-C (Hendrycks & Dietterich, 2019), large-scale and widely used benchmark for out-of-distribution generalization. It contains 15 types of 4 main categories (noise, blur, weather, digital) corrupted images and each type has 5 severity levels. We compare our SAR with the following state-of-the-art methods. DDA (Gao et al., 2022) performs input adaptation at test time via diffusion model. MEMO (Zhang et al., 2022) minimizes marginal entropy over different augmented copies w.r.t. given test sample. Tent (Wang et al., 2021) and EATA (Niu et al., 2022a) are two entropy based online fully test-time adaptation (TTA) methods.
Method
GPU
MFMO (Zhang et al., 2022)
55,980 soes
DDA (Gáo et al., 2022)
146.220 sees
TTT (Sun et 2020)
3,600
(Wang al.. 202.1
110
EATA (Niu et al., 2022a)
114 socs
SAR (ours)
115 sess
Models and Implementation Details. We conduct experiments on ResNet50- BE850-BN/GN-TAI and VitBase- LN that are obtained from torchvis or t iimm (Wightman, 2019). For our SAR, we use SGD as the update rule, with momentum of 0.9, batch size of 64 (except for the experiments of batch size=1), and learning rate of 0.00025/0.001 for ResNet/Vit models. The threshold Eo in Eqn. (2) is set to 0.4x In 1000 by following EATA (Niu et al., 2022a). p in Eqn. (3) is set by the default value 0.05 in Foret et al. (2021). For trainable parameters of SAR during TTA, following Tent (Wang et al., 2021), we adapt the affine parameters of group/layer normalization layers in ResNet50- GN/VitBase-L.N More details and hyper-parameter of compared methods are put in Appendix C.2.
5.1 ROBUSTNESS TO CORRUPTION UNDER VARIOUS WILD TEST SETTINGS
Results under Online Imbalanced Label Distribution Shifts, As illustrated in Section as the imbalance ratio 4maz /fmi increases, TTA degrades more and more severe. Here, we make com- parisons under the most difficult case: 9maz/ /m///m/nnnn i.e., test samples come in class order. We evaluate all methods under different corruptions via the same sample sequence for fair comparisons.
From Table 2, our SAR achieves the best results in average of 15 corruption types over ResNet50- GN and VitBase-LN, suggesting its effectiveness. It is worth noting that Tent works well for many corruption types on VitBase-LN (e.g., defocus and motion blur) and ResNet50-GN (e.g.. pixel), while consistently fails ResNet50-BN. This further verifies our observations in Section 4 that entropy minimization on LN/GN models has the potential to perform well under online imbalanced label distribution shifts. Meanwhile, Tent also suffers from many failure cases, e.g., VitBase-LN on shot noise and snow. For these cases, our SAR works well. Moreover, EATA has fewer failure cases than Tent and achieves higher average accuracy, which indicates that the weight regularization is somehow able to alleviate the model collapse issue. Nonetheless, the performance of EATA is still inferior to our SAR, e.g., 49.9 49.9% vs. 58.0% (ours) on VitBase-LY regarding average accuracy.
7

---

Published as a conference paper at ICLR 2023
Table 2: Comparisons with state-of-the art methods on Image.Net-C (severity level 5) under ONLINE IMBALANCED LABEL SHIFTS (imbalance ratio = 20) regarding Accuracy ( (%). "B""'"GN''"L"" is short for Batch/Group/Laay normalization. The bold number indicates the best result.
Weather
Diginl
Mothle
Cauas.
Shoc
Impul.
Defoc.
Glass
Motion
Zoom
Frost
Fog
Belt.
Coan.
IPEG
Avg.
ResNer50 (iN)
2.2
2.9
18
17.8
9.3
14.5
22.5
16.8
23.4
24.6
59.0
5.5
17.1
20.7
31.6
18.0
MEMO
19.8
13.2
20.8
27.5
28.6
32.3
60.8
11,0
23.8
37.7
24.0
TDPAA
14.6
16.4
25.5
17.
51.2
27.3
1.4
1.0
0.9
1.2
2.6
1.8
3.6
5.0
0.5
2.6
2.1
EATA
0.3
0.3
0.2
0.2
0.5
0.9
0.3
0.9
1.8
3.5
0.2
0.8
1.2
0.9
0.9
ReeNer50 (GN)
17.9
19.7
11.3
21.3
47.4
13.6
69.2
52.3
R0.6
MILMO
17.1
12.7
21.8
26.9
69.6
36.4
DIM
16.5
19.4
21,9
26.3
40.2
13.7
61.3
25.2
37.3
46.9
35.1
Tent
2.6
3.3
2.7
13.9
7.9
19.5
17.0
16.5
21,9
1,8
70.5
42.2
6.6
49.8
$3.7
22.0
FATA
27.0
28.3
25.1
14.9
17.1
24.4
25.3
32.2
$2.0
39.8
66.7
33.6
24.5
43.9
38.4
31.6
SAR (ours)
33.84:a
36.5a14
35.5.11
19.2
19:5,12
33.3.45
27.7.um
23.9
45.3am
S8.14u
71.9..)
46.7
7.1
$2.1.45
S6.3.11
37.2.m..
ValBas (LNN)
9.4
3.3
19.1
23.4
34.0
27.0
15.8
26.3
47.4
54.7
43.9
44.5
47.6
29.9
* MEMO
21.6
17.4
20.6
57.1
29.6
40.6
34.4
25.0
34.8
55.2
65.0
$4,9
37.4
55.5
$7.7
39.1
* DDA
41.3
41.3
40.6
24.6
27.4
10.7
26.9
16.2
27.7
34.8
50.0
32.5
42.2
53.5
52.7
36.2
Tent
32.7
1.4
34.6
54.4
52.3
58.2
52.2
7.7
12.0
69.3
76.1
66.1
56.7
09.4
66.4
47.3
FATA
35.9
34.6
36.7
45.3
47.2
49.3
47.7
56.5
55.4
62.2
72.2
21.7
56.2
64.7
63.7
49.9
55.3.1
54.3.0
56.9..
(8.7
76.2.c.
68.9.
58.0
Table 4: Comparisons with state-of-the-art methods on ImageNet-C (severity level 5) with BATCH SIZE=1 regarding Accuracy ( (%). "BN"I"GN"'"L" is short for Batch/Group/Layer normalization.
Noise
Weather
Digitl
Modell
Gauas.
Shot
Impul.
Defoc.
Glasss
Motion
Zoom
Snow
Frost
Fog.
Belt.
Couts.
Elastic
IPEG
Avg.
ResNed50 (BN)
2.9
1.9
17.9
9.3
14.8
22.5
16.9
23.3
25.4
58.9
5.4
17.0
20.6
31,6
18.0
MEMO
19.7
13.00
28.7
32.2
60.9
23.8
37.5
23.0
14.7
17.2
53.1
27.3
*
0.1
0.1
0.1
0.2
0.3
0.2
0.1
0.1
0.1
EADA
0.1
0.1
0.1
0.1
0.1
0.1
0.2
0.2
0.2
0.1
0.2
0.1
0.1
0.2
0.1
0.1
ReeNer50 (GN)
17.9
19.8
11.4
21.4
47.3
33.6
30.6
MEMO
17.1
12.6
21.8
26.9
47.0
34.4
69.5
36.5
31.2
DEA
42.3
16.6
19.6
31.9
26.0
40:1
13.7
61.2
25.2
* Temt
2.5
2.9
2.5
13.5
3.6
18.6
17.6
15.3
23.0
1.4
70.4
43,2
6.2
49.2
$3.8
21.5
* FATA
24.8
28.3
25.7
18.1
17.3
28.5
20.3
44-5
44.3
41.6
20.9
44.6
22.0
46.8
55.7
36.5
* SAR (ours)
23.4.aa
26.6.11
23.9.4:
18:4
15,4ana
28.6..33
30.4.s:
$4.9
44.7443
25.7am
72.3.cs
44.5
14.8a31
47.0...
$6.1uan
34.5
VitBase (L.N)
9.5
6.7
8.2
20.0
23.4
33.9
27.1
15.9
26.5
47.2
54.7
44.1
30.5
44.5
47.8
29.5
* MEMO
21.6
17.3
20.6
37.1
29.6
40.4
34.4
24.9
34.7
55.1
64.8
54,9
37.4
58.4
$7.6
34.1
DEM,
41.3
41.1
40.7
24.4
27.2
30.6
26.9
18.3
27.5
34.6
50,1
32.4
42.3
52.2
52.6
36.1
Tent
42.2
1.0
43.3
$2.4
48.2
55.5
511.
16.5
16.9
66.4
74,9
64.7
51.6
07.0
64.3
47.7
EATA
29.7
25:1
$4.6
44.7
39.2
48.3
42.4
37.5
45.9
60.0
65.9
61.2
46.4
58.2
59.6
46.6
53.7.13
50.7.m
681.
74.8.23
65.7.ra
57.9
56.3.11
Results under Mixed Distribution Shifts. We evaluate dif- ferent methods on the mixture of 15 corruption types (total of 15x50,000 images) at different severity levels (5&3). From Table 3, our SAR performs best consistently regarding accu- racy, suggesting its effectiveness. Tent fails (occurs collapse) on ResNet50-GN levels 5&3 and VitBase-LN level 5 and thus achieves inferior accuracy than no adapt model, showing the
Table 3: Comparisons with state-
of-the-arts on ImageNet-C under MIXTURE OF 15 CORRUPTION TYPES regarding Accuracy ( (%).
Model Method
Level
Level
ResNer50
18.0
39.7
MEMO (Zhing al., 2002)
23.9
46.2
DDD (Gao 2022)
27.3
44.2
instability of long-range online entropy minimization. Com-
Teax (Wang et 2021)
2.3
41.1
pared with Tent, although MEMO and DDA achieve better re-
EATA (Niu al. 20226)
26.8
52.6
ResNer50 (GN)
30.6
54.0
sults, they rely on much more computation (inefficient at test
MEMO (2hang et al., 2002)
31.2
54,5
time) as in Table 1, and DDA also needs to alter the train-
DDD (Laao et al. 2022)
Tost (Wang all 3021)
35.1
13.4
52.3
33,1
ing process (diffusion model training). By comparing Tent and EATA (both of them are efficient entropy-based) EATA achieves good results but it needs to pre-collect a set of in-
EATA (Niu 2022c)
38.1
56.1
(ours)
38.3103
57.4
VilBase (LN)
29.9
53.8
MEMO (2hng et al., 2012)
39.1
62.1
DDA (Gao et 2022)
36.1
53.2
distribution test samples (2,000 images in EATA) to compute Fisher importance for regularization and then adapt, which sometimes may be infeasible in practice. Unlike EATA, our SAR does not need such samples and obtains better results than EATA.
Teax (Wang et 2021)
16.5
70.2
EATA (Niu al., 2022a)
55.7
69.5
SAR tours)
57.1
70.7
Results under Batch Size = 1. From Table 4, our SAR achieves the best results in many cases. worth noting that MEMO and DDA are not affected by small batch sizes, mix domain shifts, or online imbalanced label shifts. They achieve stable/same results under these settings since they reset (or fix) the model parameters after the adaptation of each sample. However, the computational complexity of these two methods is much higher than SAR (see Table 1) and only obtain limited performance gains since they cannot exploit the knowledge from previously seen images. Although EATA performs better than our SAR on ResNet50-GN, it relies on crecooleeting 2,000 additional in-distribution samples (while we do not). Moreover, our SAR consistently outperforms EATA in other cases, see batch size results on VitBase-INN, Tables 2-3, and Tables 8-9 in Appendix.
8

---

Published as a conference paper at ICLR 2023
Table 5: Effects of components in SAR. We report the Accuracy (%) on ImageNet-C (level 5) under ONLINE IMBALANCED LABEL SHIFTS (imbalance ratio qman/ Qmin oO). "reliable" and "sharpness-awar" aware (sa)" denote Equ. (2) and Equ. (3), "recover" denotes the model recovery scheme.
Noise
Blur
Weather
Digima
Misiel-Method
Gaues.
Impul.
Defoc.
Glass
Metion
7oom
Snow
Fog
Contr.
Flastic
Piscl
JPFC
Aug.
ResNec50 Enioon
17.1
8.5
27.0
17.9
45.8
52.2
+ reliable
34.5
3688
36.2
19.5
3.1
33.6
14.5
20.5
38.3
244
71.9
47.0
8.3
52.1
564
31.7
rellibleess
33.8
35.9
36.4
19.2
18.7
33.6
24.5
23.5
45.2
49.3
71.9
46.6
9.2
51.6
56.4
37.0
33.6
36.1
36.2
19.1
18.6
33.9
22.5
71.9
46.5
9.2
51.5
ViiRae Nyeeotoopy
21.2
38.6
54.8
52.7
58.5
10.1
69.6
66.3
59.2
60.7
66.8
47.6
+ reliabble
47.8
35.7
48.4
55.2
54.1
58.6
51.1
13.3
21.1
69.5
76.2
66.1
60.2
69.3
66.7
53.1
reliubleess
47.9
47.6
48.5
55.4
54.2
58.8
54.6
19.7
22.1
09.4
76.3
66.2
00.9
69.4
06.6
54.5
47.9
55.4
543
58.8
49.1
48.3
69.4
66.2
60.9
60.4
5.2 ABLATE EXPERIMENTS
Clip by Value
Clip by Norm
Online Accurccy (S6
40-
40-
Comparison with Gradient Clipping, As
mentioned in Section 3.2, gradient clip-
ping is straightforward solution to allevi- ate model collapse. Here, we compare our SAR with two variants of gradient clip, i.e., by value and by norm. From Figure 6, for both two variants, it is hard to set proper threshold 5 for clipping, since the gradients for different models and test data have dif- ferent scales and thus the ö selection would be sensitive. We carefully select 8 on specific test set (shot noise level 5). Then, we select a very small ó to make gradient clip work, i.e., clip by value 0.001 and by norm 0.1. Nonetheless, the per- formance gain over "no adapt" is very marginal, since the small 8 would limit the learning ability of the model and in this case the clipped gradients may point in very different direction from the true gradients. However, a large fails to stabilize the adaptation process and the accuracy will degrade after the model collapses (e.g.. clip by value 0.005 and by norm 1.0). In contrast, SAR does not need to tune such a parameter and achieve significant improvements than gradient clipping.
20-
SAB Ioumn
20
0
500
1000
1500
500 1000
1500
Online Batch
Online Batch
Figure Comparison with gradient clipping. Re- sults on VitBase-LN, ImageNet-C, shot noise, sever- ity level 5, online imbalanced (ratio = oo) label shift. Accuracy is calculated over all previous test samples.
Effects of Components in SAR. From Table 5, compared with pure entropy minimization, the reliable entropy in Eqn. (2) clearly improves the adaptation performance, i.e., 47.6% + 53.1% on VitBase-LN and 24.6% 31.7% on ResNet50-GN w.r.t. average accuracy. With sharpness-aware (sa) minimization in Eqn. (3), the accuracy is further improved, e.g., 31.7% -37.0% on ResNet50- GN w.r.t. average accuracy. On VitBase-LN, the sa module is also effective, and it helps the model to avoid collapse, egg., 35.7% + 47.6% on shot noise. With both reliable and sa, our method performs stably except for very few cases, i.e., VitBase-LN on snow and frost. For this case, our model recovery scheme takes effects, i.e., 54.5% + 58.2 58.2% on VitBase-LN w.r.t. average accuracy.
Sharpness of Loss Surface. We visualize the loss surface by adding perturbations to model weights, as done in Li et al. (2018b). We plot Figure 7 via the model weights obtained after the adaptation on the whole test set. By comparing Figures 7 (a) and (b), the area (the deepest blue) within the lowest loss contour line of our SAR is larger than Tent, showing that solution is flat- ter and thus is more robust to noisy/large gradicnts. learned on ImageNet-C Gaussian noise, level 5.
[a] VilBase-LN, Teni
(b) VitBase-IN, SAR (Ours)
6.2
6.2
90
4.8 90
4.8
60-
3.6 60
3.6
2.4
2.4
30-
1.2
30
1.2
0.0
0.0
30
60
90
30
60
90
Figure 7: Loss (entropy) surface. Models
CONCLUSIONS
In this paper, we seek to stabilize online test-time adaptation (TTA) under wild test settings, iee, mix shifts, small batch, and imbalanced label shifts. To this end, we first analyze and conduct extensive empirical studies to verify why wild TTA fails. Then, we point out that batch norm acts as a crucial obstacle to stable TTA. Meanwhile, though batch-agnostic norm (i.e., group and layer norm) performs more stably under wild settings, they still suffer from many failure cases. To address these failures, propose sharpness-awart aware and reliable entropy minimization method (SAR) by suppressing the effect of certain noisy test samples with large gradients. Extensive experimental results demonstrate the stability and efficiency of our SAR under wild test settings.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
This work was partially supported by the Key Realm R&D Program Guangzhou 202007030007, National Natural Science Foundation of China (NSFC) 62072190, Ministry of Science and Tech- nology Foundation Project 2020AAA0106900 Program for Guangdong Introducing Innovative and Enterpreneurial Teams 20172T07X183 183, CCF-Tencent Open Fund RAGR20220108.
REPRODUCIBILITY STATEMENT
In this work, implement all methods (all compared methods and our SAR) with different mod- els (ResNet50-BN, ResNe150-GN, VitBase-LN) on the ImageNet-C/R and VisDA-2021 datasets. Reproducing all the results in our paper depends the following three aspects:
1. DATASET. The first paragraph of Section and Appendix C.1 provide the details of the adopted datasets and the download url.
2. MODELS. All adopted models (with the pre-trained weights) for test-time adaptation are publicly available. Specifically, ResNet50-BN is from torchvis s ion, ResNet50-Gi and VitBase-L.N from t im repository (Wightman, 2019). Appendix C.2 provides the download ur 1 of them.
3. PROTOCOLS OF EACH METHOD. The second paragraph of Section 5 and Appendix C.2 pro- vides the implementation details of all compared methods and our SAR. We reproduce all com- pared methods based on the code from their official GitHub, for which the download url is provided (in Appendix C.2) following each method introduction. The source code of SAR has been made publicly available.
REFERENCES
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint 160v:0600,006500 2016.
Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Josh Tenenbaum, and Boris Katz. Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models. In Advances in Neural Information Processing Systems, volume 32, 2019.
Alexander Bartler, Andre Bühler, Felix Wiewel, Mario Döbler, and Bin Yang. MT3: meta test- time training for self-supervised test-time adaption. In International Conference on Artificial Intelligence and Statistics, pp. 3080-3090, 2022.
Dina Bashkirova, Dan Hendrycks, Donghyun Kim, Haojin Liao. Samarth Mishra, Chandramouli Rajagopalan, Kate Saenko, Kuniaki Saito, Burhan UI Tayyab, Piotr Teterwak. et al. Visda-2021 competition: Universal domain adaptation to improve performance on out-of-distribe disttribbbti data. In NeurlPS 2021 Competitions and Demonstrations Track, pp. 66-79, 2022.
Vasudha Bhatnagar, Sharanjit Kaur, and Sharma Chakravarthy. Clustering data streams using grid- based synopsis. Knowledge and Information Systems, 117-1122 2014.
Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In IEEE Conference Computer Vision and Pattern Recognition, pp. 295-305, 2022a.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. International Conference on Machine Learning, pp. 1597-1607, 2020.
Xiangning Chen, Cho-Jui Hsich, and Boqing Gong. When vision transformers outperform resnets without pre-training or strong data augmentations. International Conference on Learning Rep- resentations, 2022b.
Sungha Choi, Seunghan Yang, Scokeon Choi, and Sungrack Yun. Improving test-time adaptation via shift-agnostic weight regularization and nearest source prototypes. In European Conference Computer Vision, 2022.
10

---

Published as a conference paper at ICLR 2023
Yunjey Choi, Minje Choi, Munyoung Kim, Jung- Woo Ha, Sunghun Kim, and Jaegul Choo. Stargan: Unified generative adversarial networks for multi-domain image-to to-maago translation. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 8789-8797, 2018.
Sayak Ray Chowdhury and Aditya Gopalan. On kernelized multi-armed bandits. In International Conference on Machine Learning, pp. 844-853, 2017.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 248-25422 205,225,220 2009.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. In Ad- vances Neural Information Processing Systems, pp. 8780-8794, 2021.
Qi Dou, Daniel Coelho de Castro, Konstantinos Kamnitsas, and Ben Glocker. Domain general- ization via model-agnostic learning of semantic features. In Advances in Neural Information Processing Systems, pp. 6447-6458, 2019.
Jiawei Du, Hanshu Yan, Jiashi Feng, Joey Tianyi Zhou, Liangli Zhen, Rick Siow Mong Goh, and Vincent Tan. Efficient sharpness-awar aware minimization for improved training of neural networks. In International Conference on Learning Representations, 2022.
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimiza- tion for efficiently improving generalization. International Conference on Learning Represen- tations, 2021.
Jin Gao, Jialing Zhang, Xihui Liu, Evan Shelhamer, Trevor Darrell, and Dequan Wang. Back to the source: Diffusion-driven test-time adaptation. In ICML Workshop on Updatable Machine Learning, 2022.
Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. In International Conference on Learning Representations, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
Dan Hendrycks and Thomas Dictterich. Benchmarking neural network robustness to common cor- ruptions and perturbations. In International Conference on Learning Representations, 2019.
Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. Augmix: A simple data processing method to improve robustness and uncertainty. In International Conference on Learning Representations, 2020.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang. Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. IEEE International Conference on Computer Vision, pp. 8320-8329 2021.
Sepp Hochreiter and Jürgen Schmidhuber. Flat minima. Neural computation, 9(1):1-42 1997.
Steven CH Hoi, Doyen Sahoo, Jing Lu, and Peilin Zhao. Online learning: A comprehensive survey. Neurocomputing, 459:249-225 28999228922290 2021.
Junyuan Hong. Lingjuan Lyu, Jiayu Zhou, and Michael Spranger. MECTA: Memory-cconomic
continual test-time model adaptation. International Conference on Learning Representattinss, 2023.
Xuefeng Hu, Gokhan Uzunbas, Sirius Chen, Rui Wang, Ashish Shah, Ram Nevatia, and Ser-Nam Lim. Mixnorm: Test-time adaptation through online normalization estimation. arXiv preprint arXiv:2110.11478, 2021.
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning, pp. 448-456, 2015.

---

Published as a conference paper at ICLR 2023
Yusuke Iwasawa and Yutaka Matsuo. Test-time classifier adjustment module for model-agnostic domain generalization. In Advances in Neural Information Processing Systems, pp. 2427-24401 2021.
Ansh Khurana, Sujoy Paul, Piyush Rai, Soma Biswas, and Gaurav Aggarwal. SITA: single image test-time adaptation. arXiv preprini arXiv:2112.02355. 2021.
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Bal- subramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, al. WILDS: A benchmark of wi-lhe-wild distribution shifts. In International Conference on Machine Learning, pp. 5637-5664, 2021.
Jogendra Nath Kundu, Naveen Venkat, Venkatesh Babu, et al. Universal source-free domain adaptation. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 4544 4553, 2020.
Jungmin Kwon, Jeongseop Kim, Hyunseo Park, and Kwon Choi. ASAM: adaptive sharpness- aware minimization for scale-invariant learning of deep neural networks. In International Con- ference on Machine Learning, pp. 5905-5914, 2021.
Boyi Li, Felix Wu, Ser-Nam Lim, Serge Belongie, and Kilian Q Weinberger. On feature normaliza- tion and data augmentation. In IEEE Conference Computer Vision and Pattern Recognition, pp. 12383-12392, 2021.
Li Yongxin Yang. Yi-Zhe Song, and Timothy Hospedales. Learning to generalize: Meta-learning for domain generalization. In AAAI Conference on Artificial Intelligence, pp. 3490-3497, 201 8a.
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss land- scape of neural nets. In Advances in Neural Information Processing Systems, pp. 6391-6401, 2018b.
Rui Li, Qianfen Jiao, Wenming Cao, Hau-San Wong, and Si Wu. Model adaptation: Unsupervised domain adaptation without source data. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 9638 9647, 2020.
Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning, pp. 6028-6039) 2020.
Hyesu Lim, Byconggeun Kim, Jacgul Choo, and Sungha Choi. TTN: domain-shift aware batch
normalization in test-time adaptation. In International Conference on Learning Representations, 2023.
Sungbin Lim, Ildoo Kim, Taesup Kim, Chiheon Kim, and Sungwoong Kim. Fast autoaugment. In Advances in Neural Information Processing Systems, pp. 6665-6675, 2019.
2011..
Hongbin Lin, Yifan Zhang, Zhen Qiu, Shuaicheng Niu, Chuang Gan, Yanxia Liu, and Mingkui Tan. Prototype-guided continual adaptation for class-incremental unsupervised domain adaptation. In European Conference on Computer Vision, pp. 351-368 2022.
Yuejiang Liu, Parth Kothari, Bastien van Delft, Baptiste Bellot-Gurlet, Taylor Mordan, and Alexan- dre Alahi. TTT++: when does self superviedd test-time training fail or thrive? In Advances in Neural Information Processing Systems, volume 34, pp.
2021.
Zhuang Liu, Hanzi Mao, Chao-Yuse Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 11976-11986, 2022.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. International Confer- ence on Learning Representations, 2018.
Chaithanya Kumar Mummadi, Robin Hutmacher, Kilian Rambach, Evgeny Levinkov, Thomas Brox, and Jan Hendrik Metzen. Test-time adaptation to distribution shift by confidence maxi- mization and input transformation. arXiv preprint arXiv:2 14999, 2021.
12

---

Published as a conference paper at ICLR 2023
Zachary Nado, Shreyas Padhy, D Sculley, Alexander D D'mourr Balaji Lakshminarayanan, and Jasper Snock. Evaluating prediction-time time batch normalization for robustness under covariate shift. arXiv preprint arXiv:2006. 10963, 2020.
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In International Conference on Machine Learning. pp. 16888-16905, 2022a.
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Guanghui Xu, Haokun Li, Junzhou Huang, Yaowei Wang, and Mingkui Tan. Boost test-time performance with closed-loop inference. arXiv preprint arXiv:2203. 10853, 2022b.
Emin Orhan. Robustness properties of facebook's resnext WSL models. arXiv preprint 11Xiv:1907.07640. 2019.
Zhongyi Pei, Zhangjie Cao, Mingsheng Long, and Jianmin Wang. Multi-adversarial domain adap- tation. In AAAI Conference on Artificial Intelligence, pp. 3934-3941, 2018.
Zhen Qiu, Yifan Zhang, Hongbin Lin, Shuaicheng Niu, Yanxia Liu, Qing Du, and Mingkui Tan. Source-free domain adaptation via avatar prototype generation and adaptation. In International Joint Conference on Artificial Intelligence, pp. 2921-2922 2021.
Alexander Rakhlin, Karthik Sridharan, and Ambuj Tewari. Online learning: Random averages, com- binatorial parameters, and learnability. In Advances in Neural Information Processing Systems, pp. 1984-1992, 2010.
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do cifar-10 classifiers generalize to cifar-10? arXiv preprini 1006::8800.00444 2018.
Mengye Ren, Tyler R Scott, Michael L luzzolino, Michael C Mozer, and Richard Zemel. Online unsupervised learning of visual representations and categories. arXiv preprint arXiv:2109.05675 2021.
Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. Maximum classifier dis- crepancy for unsupervised domain adaptation. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 3723-3732, 2018
Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. In Advances in Neural Information Processing Systems, volume 33, pp. 11539-11551, 2020.
Shai Shalev-Shwatzz al. Online learning and online convex optimization. Foundations and Trends@ in Machine Learning, 4(2):107-194, 2012.
Shiv Shankar, Vihari Piratla, Soumen Chakrabarti, Siddhartha Chaudhuri, Preethi Jyothi, and Sunita
Sarawagi. Generalizing across domains via cross-gradient training. International Conference on Learning Representations, 2018.
Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time train- ing with sulf--uuee for generalization under distribution shifts. In International Conference Machine Learning. pp. 9229-9248. 2020.
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Repre- sentations, 2021.
Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In IEEE Conference Computer Vision and Pattern Recognition, pp. 7201-7211, 2022.
Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. Non-local neural networks. In
IEEE Conference on Computer Vision and Pattern Recognition, pp. 7794-7803, 2018.
Ross Wightman. Pytorch image models.
https://github.e com wtt.om/wwwghhtee
pytorch-
getorch-immge-mode imag--mddeeeeeeee 2019.
13

---

Published as a conference paper at ICLR 2023
Yuxin Wu and Kaiming He. Group normalization. In European Conference Computer Vision, pp. 3-19, 2018.
Yuxin Wu and Justin Johnson. Rethinking "batch" in batchnorm. arXiv preprint arXiv:2105.07574 2021.
Huaxiu Yao, Yu Wang, Sai Li, Linjun Zhang, Weixin Liang, James Zou, and Chelsea Finn. Im- proving out-of-distribution robustness via selective augmentation. In International Conference on Machine Learning, pp. 25407-25437, 2022.
Chen Zhang and Steven CH Hoi. Partially observable multi-sensor sequential change detection: A combinatorial multi-armed bandit approach. In AAAI Conference Artificial Intelligence, pp- 5733-5740, 2019.
Marvin Zhang, Sergey Levine, and Chelsea Finn. Memo: Test time robustness via adaptation and augmentation. Advances in Neural Information Processing Systems, 2022.
Yifan Zhang, Peilin Zhao, Shuaicheng Niu, Qingyao Wu, Jiezhang Cao, Junzhou Huang, and Mingkui Tan. Online adaptive asymmetric active learning with limited budgets. IEEE Trans- actions on Knowledge and Data Engineering, 33(6):2680-266) 2019.
Yifan Zhang. Shuaicheng Niu, Zhen Qiu, Ying Wei, Peilin Zhao, Jianhua Yao, Junzhou Huang, Qingyao Wu, and Mingkui Tan. COVID-DA: Deep domain adaptation from typical pneumonia to covid-19. arXiv preprint arXiv:2005.01577 2020a.
Yifan Zhang, Ying Wei, Qingyao Wu, Peilin Zhao, Shuaicheng Niu, Junzhou Huang, and Mingkui Tan. Collaborative unsupervised domain adaptation for medical image diagnosis, IEEE Transac- tions on Image Processing, 29:7834-7844, 2020b.
Bowen Zhao, Chen Chen, and Shu-Tao Xia. DELTA: Degradation-fice fully test-time adaptation. In International Conference on Learning Representations, 2023.
Peilin Zhao and Steven CH Hoi. Cost-sensitive online active learning with application to mali- cious URL detection. In Proceedings of ACM SIGKDD International Conference Knowledge Discovery and Data Mining, pp. 919-927, 2013.
Peilin Zhao, Steven CH Hoi, and Rong Jin. Double updating online learning. Journal of Machine Learning Research, 12:1587, 2011.
Yaowei Zheng, Richong Zhang, and Yongyi Mao. Regularizing neural networks via adversarial model perturbation. In IEEE Conference Computer Vision and Pattern Recognition, pp. 8156- 8165, 2021.
14

---

Published as a conference paper at ICLR 2023
APPENDIX
CONTENTS
Related Work
16
Pseudo Code of SAR
18
More Implementation Details
19
C.I More Details on Dataset
19
C.2 More Experimental Protocols
19
Additional Results ImageNet-C of Severity Level 3
22
D.I Comparisons with Ktateoof-the-at under Online Imbalanced Label Shift
22
D.2 Comparisons with State-of-the-ard under Batch Size
22
Additional Results on ImageNet-R and VisDA-2021
23
Additional Ablate Results
24
E.I Effects of Components in SAR
24
F2 Visualization of Loss Surface Learned by SAR
25
E3 Sensitivity of p SAR
25
G Additional Discussions
26
G.1 More Discussions Model Collapse
26
G.2 Effects Large Batch Sizes BN models under Mix Domain Shifts
................................
26
G.3 Performance of Tent with ConvNeXt-L.N
26
G.4 Effectiveness of Model Recovery Scheme with Tent and EATA
26
15

---

Published as a conference paper at ICLR 2023
A RELATED WORK
We relate our SAR to existing adaptation methods without and with target data, sharpness- aware optimization, online learning methods, and EATA (Niu et al., 2022a).
Adaptation without Target Data. The problem of conquering distribution shifts has been stud- ied in a number of works at training time, including domain generalization (Shankar et al., 2018; Li et al., 2a; Dou et al., 2019), increasing the training dataset size (Orhan, 2019), various data augmentation techniques (Lim et al., 2019; Hendrycks et al., 2020; Li et al., 2021; Hendrycks et al., 2021; Yao et al., 2022), to name just new. These methods aim to pre-anticipate or simulate the possible shifts of test data at training time, so that the training distribution can cover the possible shifts of test data. However, pre-anticipating all possible test shifts training time may be infea- sible and these training strategies are often more computationally expensive. Instead of improving generalization ability at training time, we conquer test shifts by directly learning from test data.
Adaptation with Target Data. We divide the discussion on related methods that exploit target data into 1) unsupervised domain adaptation (adapt offline) and 2) test-time adaptation (adapt online).
Unsupervised domain adaptation (UDA), Conventional UDA jointly optimizes on the labeled source and unlabeled target data to mitigate distribution shifts, such as devising a domain discrimi- nator to align source and target domains at feature level (Pei et al., 2018; Saito et al., 2018; Zhang et al., 2020b;a) and aligning the prototypes of source and target domains through a contrastive learn- ing manner (Lin et al., 2022). Recently, source-free UDA methods have been proposed to resolve the adaptation problem when source data are absent, such as generative- based methods that generate source images or prototypes from the model (Li et al., 2020; Kundu et al., 2020; Qiu et al., 2021), and information maximization (Liang et al., 2020). These methods adapt models on a whole test set, in which the adaptation is offline and often requires multiple training epochs, and thus are hard to be deployed on online testing scenarios.
Test-time adaptation (TTA). According to whether alter training. TTA methods can be mainly categorized into two groups i) Test-Time Training (TTT) (Sun et al., 2020) jointly optimizes a source model with both supervised and self-supervised losses, and then conducts self-supervised learning at test time. The self-supervised losses can be rotation prediction (Gidaris et al.. 2018) in TTT or contrastive-based objectives (Chen et al., 2020) in TTT++ (Liu et al., 2021) and MT3 (Bartler al., 2022), etc. ii) Fully Test-Time Adaptation (Wang et al., 2021; Niu et al., 2022a; Hong et al., 2023) does not alter the training process and can be applied to any pre-trained model, including adapting the statistics in batch normalization layers (Schneider et al., 2020; Hu et al., 2021; Khurana et al., 2021; Lim et al., 2023; Zhao et al., 2023), unsupervised entropy minimization (Wang et al., 2021; Niu et al., 2022a; Zhang et al., 2022), prediction consistency maximization (Zhang et al., 2022; Wang et al., 2022; Chen et al., 2022a), top-k classification boosting (Niu et al., 2022b), etc.
Though effective handling test shifts, prior TTA methods are shown to be unstable in the online adaptation process and sensitive to when test data are insufficient (small batch sizes), from mixed domains, have imbalanced and online shifted label distribution (see Figure 1). Here, it worth not- ing that methods like MEMO (Zhang et al., 2022) and DDA (Gao et al., 2022) are not affected under the above 3 scenarios, since MEMO resets the model parameters after each sample adaptation and DDA performs input adaptation via diffusion (in which the model weights are frozen during test- ing). However, these methods can not exploit the knowledge learned from previously seen samples and thus obtain limited performance gains. Moreover, the heavy data augmentations and diffusion in MEMO and DDA are computationally expensive and inefficient at test time (see Table 6). In this work, we analyze why online TTA may fail under the above practical test settings and propose associated solutions to make TTA stable under various wild test settings.
Sharpness-aware Minimization (SAM). SAM (Foret et al., 2021) optimizes both a supervised ob- jective (e.g., cross-entropy) and the sharpness of loss surface, aiming to find a flat minimum that has good generalization ability (Hochreiter & Schmidhuber, 1997). SAM and its variants (Kwon et al., 2021: Zheng et al., 2021; Du et al., 2022; Chen et al., 2022b) have shown outstanding performance on several deep learning benchmarks. In this work, when analyze the failure reasons of test-time entropy minimization, we find that some noisy samples that produce gradients with large norms harm the adaptation and thus lead to model collapse. To alleviate this, we propose minimize the
16

---

Published as a conference paper at ICLR 2023
sharpness of the test-time entropy loss surface that the online model update is robust to those noisy/large gradients.
Online Learning (OL). OL (Hoi et al., 2021; Chowdhury & Gopalan, 2017; Zhao et al., 2011) conducts model learning from a sequence of data samples one by one at a time, which com- mon in many real-world applications (e-g., social web recommendation) According to the super- vision type, OL can be categorized into three groups: i) Supervised methods (Rakhlin et al., 2010; Shalev-Shwartz al., 2012) obtain supervision at the end of each online learning iteration, ii) Semi- supervised methods (Zhang & Hoi, 2019) can obtain supervision from only partial samples, e.g., online active learning (Zhao & Hoi, 2013; Zhang et al., 2019) selects informative samples to query the ground-trute label for the model update, iii) Unsupervised methods (Bhatnagar et al., 2014) can not obtain any supervision during the whole online learning process. In this sense, test-time adapta- tion (TTA) (Sun et al., 2020; Wang et al., 2021) online updates models with only unlabeled test data and thus falls into the third category. However, unlike unsupervised OL that mainly aims learn representations or clusters (Ren et al., 2021), TTA seeks to boost the performance of any pre-trained model on out-of-distribution test samples.
Comparison with EATA (Niu et al., 2022a). Although both EATA and our SAR include step to remove samples via entropy, their motivations behind this step are different. EATA seeks to improve the adaptation efficiency via sample entropy selection. In our SAR, discover that some noisy gradients with large norms may hurt the adaptation and thus result in model collapse under wild test settings. To remove these gradients, helps to remove partial noisy gradients with large norms, However, this is still insufficient for achieving stable TTA (see ablation results in Table 5). Thus, we further introduce the sharpness- aware optimization and a model recovery scheme. With these three strategies, our SAR performs stably under wild test settings.
exploit alternative metric (i.e., entropy), which
17

---

Published as a conference paper at ICLR 2023
B PSEUDO CODE OF SAR
In this appendix, we provide the pseudo-code of our SAR method. From Algorithm 1, for each test sample Xj. we first apply the reliable sample filtering scheme (refer lines 3-6) to determine whether it will be used to update the model. If x, is reliable, we will optimize the model via the sharpness-awe aware entropy loss x, (refer to lines 7-10). Specifically, we first calculate the optimal weight perturbation é()) based on the gradient [xy;; ④), and then update the model with ap- proximate gradients& g Exz; O) (+a(ē)- Lastly, we exploit recovery scheme to enable the model work well even under few extremely hard cases (refer to lines 11-13). Specifically, when the moving average value 6m of entropy loss is smaller than eo (indicating that the model occurs to collapse), we will recover the model parameters θ to its original/initial value.
Algorithm 1: Sharpness-Awant and Reliable Test-Time Entropy Minimization (SAR)
Input: Test samples Dtest {x,}}=1,
JM model fe(-) with trainable parameters Ô O, step size vj 0, neighborhood size p > 0, Fo > 0 Eqn. (2), 6o > for model recovery.
Output: Predictions {ūs
Initialize 6,, = è, moving average of entropy em 0;
for X; E Drest do
Compute entrrpy E,=B(x;; θ) and predict ū== fxx):::)):
if Fg Fig then continue: end
1/ reliable entropy minimization (Ren. 2)
Compute gradient V Vgb(x,;;))
Compute ē()) per Eqn. (4);
Compute gradient approximation: E 0)e+a(t))
Update 10
// sharpness minimization (Egn.
11
em +0.1xxx.0011xx (+é(()) if e.. else 0+(x)))(0)) // moving average
12
em < 6o then
Recover model weights: θ < Oo:
// model recovery
14 end
15 end
18

---

Published as a conference paper at ICLR 2023
C MORE IMPLEMENTATION DETAILS
C.I MORE DETAILS ON DATASET
In this paper, we mainly evaluate the out-of-distribution generalization ability of all methods on a large-scale and widely used benchmark, namely ImageNet-C" (Hendrycks & Dietterich, 2019). ImageNet-C constructed by corrupting the original ImageNet (Deng et al., 2009) test set. The corruption (as shown in Figure 8) consists of 15 different types, i.e., Gaussian noise, shot noise, impulse noise, defocus blur, glass blue, motion blur, zoom blur, snow, frost, fog, brightness, con- trast, elastic transformation, pixelation, and JPEG compression, in which each corruption type has different severity levels and the larger severity level means more severe distribution shift. Then, we further conduct experiments et al., 2022) to verify the effectiveness of our method. ImageNet-R contains 30,000 images with various artistic renditions of 200 ImageNet classes, which primarily collected from Flickr and filtered by Amazon MTurk annotators. VisDA-2021 collects images from ImageNet-O/Re, and OB- jectNet (Barbu et al., 2019). The domain shifts in VisDA-2021 include the changes in artistic visual styles, textures, viewpoints and corruptions.
ImageNet-R (Hendrycks al., 2021) and VisDA-2021 (Bashkirova
Gaussian Noise
Shot Noise
Impuise Noise
Defocus Blur Frosted Glass Blu
Fog
Brightness
JPEG
Figure 8: Visualizations of different corruption types in ImageNet corruption benchmark, which are taken from the original paper of ImageNet-C (Hendrycks & Dietterich, 2019).
C.2 MORE EXPERIMENTAL PROTOCOLS
All pre-trained models involved in our paper for test-time adaptation are publicly available, includ- ing ResNe150-BN obtained from torchvisionn library, ResNet-50-GN and VitBase-LN* and ConvNeXt-LNF obtained from timm repository (Wightman, 2019). We summarize the detailed characteristics of all involved methods Table and introduce their implementation details in the following.
SAR (Ours). We use SGD as the update rule, with a momentum of 0.9, batch size 64 (except for the experiments of batch size 1). and learning rate of 0.00025/0.00/ for ResNet/Vit models. The learning rate for batch size 1 set to (0.00025/16) for ResNet models and (0.001/32) for Vit models. The threshold Eg in Eqn. (2) is set to 0.4x In 1000 by following EATA (Niu et al., 2022a). in Eqn. (3) is set by the default value 0.05 in Foret et al. (2021). For model recovery, we record the
v0. ortr=-swghhh///ennd00ggga2222222
0. http://sarrrg.ooooa..ree./lliiea
01 0l-res_244nnp
hhtpp:://l.. /https://dl.fbbaiphbl icfiiiiiiiiiic les.ille..... http:///aappppaaacecc//c////ooea
19

---

Published as a conference paper at ICLR 2023
Table 6: Characteristics of ffte-of-the-ar methods. We evaluate the efficiency of different methods with ResNet-50 (group norm) on ImageNet-C (Gaussian noise, severity level 5), which consists of totally 50,000 images. The real run time is tested via a single V100 GPU. DDA (Gao et al., 2022) pre-trains an additional diffusion model and then perform input adaptation/diifusion at test time.
Method
Need source data? Online updase!!
#Forward
#Backward
Other computation
GPU (50.000 images)
MEMO (Zhang et al., 2022)
K
50,000x65
50,000x64
AugMix (Hendrycks al., 2020)
55,980 seconds
DDA (Gio al. 2022)
X
50,000x2
0
50,000 diffusion
146,220 seconds
TTT (Sun al., 2020) 2020)
50,000x21
50,000
rotation augmentation
3,600 seconds
Tont (Wing et al., 2021)
50,000
50,000
waa
110 seconds
BATA (Niu al., 2022a)
50,000
16,196
regularizer
114 seconds
(ours)
50,000
12,710
12,710x2
Equ. (4)
seconds
entropy loss values with moving average factor of 0.9 for Cm. and the reset threshold ep set to 0.2. For learnable parameters, we only update affine parameters in normalization layers by following Tent (Wang et al., 2021). However, since the top/deep layers are more sensitive and more important to the original model than shallow layers mentioned in (Mummadi et al., 2021; Choi et al., 2022), we freeze the top layers and update the affine parameters of layer or group normalization in the remaining shallow layers. Specifically, for ResNet50-GG. that has 4 layer groups (layer1, 2, 3, 4), we freeze the layer4. For ViTBase-LNN that has 11 blocks groups (blocks1-11) we freeze blocks9, blocks 1locks11.
TTT (Sun et al., 2020). For fair comparisons, we seek to compare all methods based on the same model weights. However, TTT alters the model training process and requires the model contains a self-supervised rotation prediction branch for test-time training. Therefore, we modify TTT so that it can be applied to any pre-trained model. Specifically, given pre-trained model, we add a new branch (random initialized) from the end of middle layer (2nd layer group of ResNet- 50-GN and 6th blocks group of VitBase-LNN for the rotation prediction task. We first freeze all original parameters of the pre-trained model and train the newly added branch for 10 epochs on the original ImageNet training set. Here, we apply an SGD optimizer, with a momentum of 0.9, an initial learning rate of 0.100.005 for ResNet5O-GNNWiias--ll by decreasing factor 0. Then, we take the newly obtained model (with two branches) as the base model to perform test-time training. During the test-time training phase, use SGD as the update rule with a learning rate of 0.001 for ResNetO-GN (following TTT) and 0.0001 for VitBase-LMN LN, and the data augmentation size is set to 20 (following Niu et al. (2022a)).
and decrease it at epochs 4 and 7
Tent (Wang et al., 2021). We follow all hyper-parameters that are set in Tent unless it does not provide. Specifically, use SGD the update rule, with momentum of 0.9, batch size of 64 (except for the experiments of batch size- = I and effects of small test batch sizes (in Section 4)), and learning rate of 0.00025/0.001 for ResNet/Vit models. The learning rate for batch size is set to (0.00 (0.00025/32) for ResNet models and (0.001/64) for Vit models. The trainable parameters are all affine parameters of batch normalization layers.
EATA 8 (Niu et al., 2022a). We follow all hyper-pann parameters that are set in EATA unless it does not provide. Specifically, the entropy constant Eg (for reliable sample identification) is set to 0.1 x In 1000. The for redundant sample identification is set to 0.05. The trade-off parameter /3 for entropy loss and regularization loss is set to 2,000. The number of pre-collected in-distribution test samples for Fisher importance calculation is 2,000. The update rule is SGD, with momentum of 0.9, batch size of 64 (except for the experiments of batch size = I and effects of small test batch sizes (in Section 4)), and learning rate of 0.00025/0.000 5 for ResNet/Vit models. The learning rate for batch size = is set to (0.00025/32) for ResNet models and (0.001/64) for Vit models. The trainable parameters are all affine parameters of batch normalization layers.
MEMO" (Zhang et al., 2022). We follow all syper-parameters that are set in MEMO. Specifically, the AugMix (Hendrycks et al., 2020) as set of data augmentations and the augmentation size is set to 64. For Vit models, the optimizer is Adam W (Loshchilov & Hutter, 2018), with learning
Phttps: //gi wttp:h...oooo/aaagaaaaiiiiioo/ae
20

---

Published as a conference paper at ICLR 2023
rate 0.00001 and weight decay 0.01. For ResNet models, the optimizer is SGD, with learning rate 0.00025 and no weight decay. The trainable parameters are the entire model.
DDA (Gao et al., 2022). We reproduce DDA according to its official GitHub repository and use the default
More Details Experiments in Section 4: Normalization Layer Effects in TTA. In Section 4, we investigate the effects of TTT and Tent with models that have different norm layers under {small test batch sizes, mixed distribution shifts, online imbalanced label distribution shifts}. For each experiment, we only consider one of the three above test settings. To be specific, for experiments re- garding batch size effects (Section (1)), only tune the batch size and the fest does contain multiple types of distribution shifts and its label distribution is always uniform. For experiments of mixed domain shifts (Section 4 (2)), the test samples come from the mixture of 15 corruption types, while the batch size is 64 for Tent and I for TTT. and the label distribution of test data is always uniform. For experiments of online label shifts (Section 4 (3)), the label distribution of test data is online shifted and imbalanced, while the BS is 64 for Tent and for TTT, and test data only consist of one corruption type. Moreover, is worth noting that we re-scale the learning rate for entropy minimization (Tent) according the batch size, since entropy minimization is sensitive to the learn- ing rate and a fixed learning rate often fails to work well. Specifically, the learning rate is re-scaled (0.00025,/ /32 X BS IF BS 32 ELSE 0.00025 for ResNet models and (0.001 /64) BS for Vit models. Compared with Tent, the single sample adaptation method TTT not very sensitive to the learning rate, and thus set the same learning rate for various batch sizes. We also provide the results of TTT under different batch sizes with dynamic re-scaled learning rates in Table 7.
Table 7: Batch size (BS) effects in TTT (Sun et al., 2020) with different models (different norm layers). The learning is dynamically re-scaled by xBS. We report the accuracy (%) on ImageNet-C with Gaussian noise and severity level 5.
Model
BS=1
BS=2
BS=4
BS 8
BS=16
ResNe150-BN
21.2
23.4
23.4
24.7
ResNe150-GN
40.9
40.5
40.8
41.1
40.7
"https: stttps://gii http://hhhhhuu..o....hh///..aaee yww.b///////hhyea//////a
21

---

Published as a conference paper at ICLR 2023
D ADDITIONAL RESULTS ON IMAGENET-C OF SEVERITY LEVEL 3
D.1 COMPARISONS WITH THT--RRST UNDER ONLINE IMBALANCED LABEL SHIFT
We provide more results regarding online imbalanced label distribution shift (imbalance ratio = oo) of all compared methods in Table 8. The results are consistent with that of the main paper (severity level 5), and our SAR performs best in the average of different corruption types. noting that DDA achieves competitive results under noise corruptions while performing worse for other corruption types. The reason is that the diffusion model used in DDA for input adaptation is trained via noise diffusion, and thus its generalization ability to diffuse other corruptions is still limited.
worth
Table 8: Comparisons with sate-of-the-an art methods on ImageNet-C of severity level 3 under ON- LINE IMBALANCED LABEL DISTRIBUTION SHIFTS (imbalance ratio qmaz/ 9min oo) regarding Accuracy (%). "BN"T"G""""N" is short for Batch/Group/Layee normalization.
Dighal
ModeleMMttodd
Ganvs.
Shat
Imqul.
Defoc.
Giass
Miticos
Zoom
Saow
Fog
Rral.
Contr.
Eleatic
JPEG
Avg.
ResNec50 IBN]
27.3
25.2
25.1
37.8
16.7
37.8
35.3
35.2
32.1
46.7
69.5
46.2
55.4
46.2
59.4
39.8
MEM0
37.6
34.5
36.7
41.4
25.4
44.4
40.9
44.6
37.3
50.4
70.5
56.3
5S.7
55.2
60.9
46.
DDA
89.9
50.0
49.2
35.2
31.9
38.0
36.7
35.1
34,1
35.01
66.9
33.7
59.3
53.9
59.0
64.3
Tent
34
3.2
2.0
2.4
3.4
2.4
5.4
3.0
4.8
3.4
EATA
1.3
0.9
1.2
ResNect0 (GN)
51.5
52.9
33.]
48.4
21.2
19.8
39.3
36.
54.1
55.8
75.3
69.7
59.0
29.7
06.4
S1.1
MEMO
55.9
54.3
54.3
40.1
25.1
49.5
41.4
54.8
54.1
$7.6
75.7
70.2
60.2
61.5
66.7
54.6
DID
61:0
61.0
60.5
39.3
37.3
46.4
79.7
4777
48.1
20.9
69.9
57.8
62.8
60.1
63.8
52.4
Tent
59.1
58.6
$8.3
29.0
27.9
54.7
41.1
51.3
41.4
62.0
75.2
70.1
62.3
63.7
66.4
55.4
DATA
52.3
52.9
31.7
35.7
30.1
46.4
39.5
43.88
39.8
55.7
72.4
66.6
54.7
56,0
36.2
50.3
SAR (curs)
60
60.5an.
60.2..g
47.9
36.7
58.2.00
49.7
57.9
S3.6un
65.0ge
764449
71.0ga
67.0kan
65
65.846.1
67.6
59.9
51.5
54.7
MEMO
62.1
$7.9
61.5
62.0
49.9
46.5
75.8
79.7
72.6
70.6
DIW
59.7
53.2
59.4
43.5
43.3
50.5
41.0
34.3
34.4
55.4
65.0
64.2
64.1
63.8
62.9
$3.3
Teut
68.7
68.0
68.1
68.2
63.8
70.9
63.8
67.6
41.9
76.3
78.5
79.5
75.9
76.7
73.7
69.5
FATA
65.3
62.6
63.6
63.0
$7.1
66.3
59.5
64.5
61.0
71.3
76.9
75.9
74.2
74.8
7%.1
67.4
SAR (eUIS)
68.8-n]
68.2
68.2gm)
68
68.4212
68
1202
64
64.7
71.0g02
64.2
68. gi 66.0g.
764461
790
79.
76,2
76.2
37.1
24.1
71.3a1
D.2 COMPARISONS WITH STATE-OF- THE---RRST UNDER BATCH SIZE OF 1
We provide more results regarding batch size 1 of all compared methods in Table The results are consistent with that of the main paper (severity level 5), and our SAR performs best in the average of 15 different corruption types.
Table Comparisons with state of thethtrrrt methods on ImageNet- C of severity level 3 under BATCH SIZE=1 regarding Accuracy ( (%). "BN"/"GN" is short for Batch/Group/Layy normalization.
Dignal
Model-Mdehhod
Gauss.
Impul.
Defoc.
Motion
Zoom
Fog
Contr.
Avg.
ResNee50 (BN)
27.6
25.0
25.1
38.0
16.9
33.7
15.2
35.2
32.1
46.5
69.6
45.
55.6
46.2
59.1
39.7
MEMO
37.5
34,3
36.5
41.2
23.3
44.2
41.0
44.5
37.4
52.3
70.5
56.0
58.7
55.3
60.3
46.1
DDA
49.8
49.9
49.2
33.2
32.0
37.9
36.5
35.2
34.2
34.9
64.9
35.5
59.3
53.9
59.0
44.2
Tent
0.1
0.2
0.2
0.1
0.1
0.2
0.2
0.2
0:2
0.2
0.2
0.2
EATA
0.2
0.2
0.2
0.2
ResNect0 (GN)
54.5
52.8
53.]
48.3
21.2
89.7
39.2
50.
54.0
55.8
15.4
69.3
29.8
29.7
66.1
50D
MEMO
35.7
54.2
53.9
40.0
22.8
49.2
41.2
34.8
54.1
$7.6
75.5
69.9
60.11
61.3
66.6
5L.5
DNDA
63.0
60.9
60.4
39.2
37.2
46.4
39.7
47.7
4%.1
29.8
70.0
58.0
62.7
60.2
65.8
52.3
Text
$8.8
58.5
58.7
18.2
26.8
9
42.6
51.6
38.8
61 9
75.3
70.0
62.3
63.5
66
85.2
DATA
59.2
58.7
58.8
45.7
32.6
55.5
45.9
$6.4
52.7
63.6
75.9
71.1
64.7
64.5
67.8
58.2
SAR (curs)
60.3-n
59.0an
19.5
46.6_.11
33.0
$7.5km
47.8448
57 Mug
$2.8an
65.1gm)
76.746
71.4aa)
67.3
66.046.1
67.4gg
59.3
51.0
54.7
MEMO
61.9
57.7
61.4
61.8
49.8
46.6
73.9
75.7
79.6
72.6
72.1
61.9
DXDA
59.8
58.2
43.2
50.4
40.9
55.2
64.9
64.0
64.2
$3.2
Tent
67.1
66.2
66.3
66.3
60.9
68.1
61.4
65.2
60.4
75.2
78.1
78.8
74.9
75.8
72.4
692
EATA
60.7
53.5
61.6
60.1
$1.8
64.2
34,8
53.3
52.6
72.5
75.6
77.9
71,3
71.3
69.7
63.6
SAR (cun5)
68.5 68.5-11
67.6
67.8
68
68.0g
63.1
70.7
70.74gL
63.5
66.921
62.8g1
75. 75.8466
77.7
77.7
7844g14
74.7
35.7466
72.7
70.3.1
22

---

Published as a conference paper at ICLR 2023
E ADDITIONAL RESULTS ON IMAGENET-R AND VISDA-2021
We further conduct experiments on ImageNet-R under two wild test settings: online imbalanced label distribution shifts (in Table 10) and batch size (in Table 11). The overall results consistent with that on ImageNet-C: 1) ResNet50-GN and VitBase-LN perform more stable than ResNet50-Bar Compared with Tent and EATA, SAR achieves the best performance on ResNet50- GN and VitBase-LN.
Table 10: Comparison in terms of accuracy (%)
Comparison in terms of accuracy (%) under the wild setting single sample adaptation (batch size 1) ImageNet-R.
Table 11:
under the wild setting online imbalanced label distribution shifts ImageNet-R.
Method ResNe150 ResNet50 GN
VitBase- LN
Method ResNer50- BN ResNe150-GiN
VitBase INN
No Adapt.
36.2
40.8
43.1
No Adapt.
36.2
40.8
43.1
Tent
6.6
41.7
45.2
Tent
0.6
42.2
40.5
EATA
40.9
47.5
EATA
42.3
52.5
SAR (ours)
42.9
52.0
SAR (ours)
43,9
53.1
We also compare method with online TTA methods (Tent and EATA) on VisDA-2021. Results in Table 12 are consistent with that on ImageNet-C and ImageNet-R. Specifically, Tent and EATA fail with the BN model (ResNet50-BN) while performing stable with the GN/LN model (ResNet50- GN/VitBase-LN) under online imbalanced label shifts. Besides, our SAR further improves the adap- tation performance of Tent and EATA ResNetSO-GGWWiiase--LN
Table 12: Classification Accuracy (%)
VisDA-2021 under online inbalanced label distribu-
tion shifts and mixed domain shifts.
Model
No adapt
Tent
EATA
SAR (ours)
ResNer50-BN
36.0
9.0
7.4
ResNet50-GN
43.7
42.8
40.9
44.1
VitBase-LN
44.3
49.1
49.0
52.0
23

---

Published as a conference paper at ICLR 2023
F ADDITIONAL ABLATE RESULTS
F.1 EFFECTS OF COMPONENTS IN SAR
We further ablate the effects of components in our SAR in Figure 9 by plotting the changes of gradients norms of our methods with different components. From the results, both the reliable (Eqn. 2) and sharpness-aware (sa) (Eqn. 3) modules together ensure the model's gradients keep in normal range during the whole online adaptation process, which is consistent with our previous results in Section 5.2.
30
Gradient Norm (Avg.)
entrapy+reliable entropy-+relibbiess
20
10
500
1000
1500
Online Batch
Figure 9: The evolution of gradients during online test-time adaptation. Results VitBase, ImageNet-C, shot noise, severity level 5, online imbalance label shift (imbalance ratio 00). "reli- able" and "sharpness-aware (sa)" are short for Eqn. (2) and Eqn. (3), respectively.
We also conduct more ablation experiments under the wild test settings of "batch size (BS)=1" in Table 13 and "mix domain shifts" in Table 14. The results generally consistent with that Table 5. Both the reliable entropy and sharpness-awan optimization work together to stabilize online TTA. It is worth noting that only in VitBase-LN under BS=I the model recovery scheme is activated and improves the average accuracy from 55.7% to 56.4%.
Table 13: Effects of components in SAR. We report the Accuracy (% ImageNet-C (level 5) un- der BATCH SıZE=1. "reliable" and "sharpness-awe aware (sa)" denote Equ. (2) and Eqn. (3), "recover" denotes the model recovery scheme.
Noise
Blur
Weather
Digital
Medel-MMethod
Gauss.
Shot
Impul.
Defoc.
Glass.
Metion
Zoom
Snow
Frost
Fig
Rrit.
Contr:
Flastic
Pisel
JPRG
Avp.
ResNNc:0 Entooy
3.4
4.5
4.0
16.7
5.9
27.0
29.7
16.3
26.6
2.1
72.1
46.4
7.5
52.5
56.2
24.7
+ reliableesa
23.8
264
24.0
18.6
15.4
28.3
30.5
44.8
44.8
26.7
72.4
44,5
12.2
46.9
65.1
34.4
relibblees-nnoower
23.8
26.4
23.9
18.5
15.4
26.3
30.6
48.6
24.7
72.4
44.4
12.3
47.0
56.1
ViiRase Ny=Fntropy
64.7
67.2
reliable
34.8
4.2
35.5
50.5
45.9
51.0
52.5
47.8
65.5
63.4
51.4
65.6
63.0
* reliableesa
40.4
373
41.2
53.6
30.6
57.3
53.0
58.7
40.9
68.8
75.9
65.7
57.9
69.0
66.0
55.7
reliabll
41.2
5066
Table 14: Effects of components in SAR. We report the Accuracy (%) under MIXED DOMAIN SHIFTS, ie., mixture of 15 corruption types of ImageNet-C with severity level S. "reliable" and "sharpness-aware (sa)" denote Eqn. (2) and Eqn. (3), "recover" denotes the model recovery scheme.
Model + Method
Accuracy (%)
ResNet50 (GN)yEntropy
33.5
+ reliable
38.1
+ reliable+ss
38.2
+ reliable+ss+rrcoveer
38.2
ResNet50 (GN)pEntropy
18.5
reliable
55.2
reliable+sa
57.2
reliable+sa-rccoverr
57.2
24

---

Published as a conference paper at ICLR 2023
F.2 VISUALIZATION OF LOSS SURFACE LEARNED BY SAR
In Figure 7 in the main paper, we have visualized the loss surface of Tent and our SAR on VitBase- LN. In this section, we further provide visualizations of ResNet50-GN. We select checkpoint batch 120 to plot the loss surface for Tent, since after batch 120 this model starts to collapse. In this case, the loss (entropy) hard to degrade and cannot find proper minimum. For our SAR, the model weights for plotting are obtained after the adaptation on the whole test set. By comparing Figures 10 (a)-(b), SAR helps stabilize the entropy minimization process and find proper minima.
(a) ResNet50-GN, Tent
(b) ResNet50-GN, SAR (Ours)
6.2
6.2
90
4.8
90
4.8
60
3.6
60
3.6
2.4
2.4
30
1.2
1.2
0.0
0.0
30
90
30
60
90
Figure 10:
Visualization of loss (entropy) surface. Models are learned on ImageNet-C of Gaussian noise with severity level
F.3 SENSITIVITY OF P IN SAR
The hyper-parameter p (in Eqn. (3)) in sharpness-aware optimization is hard to tune in our online test-time adaptation. Following Foret et al. (2021), we set 0.05 in all experiments, and it works well with different model architectures (ResNet50-BN, ResNe150-GN, VitBase-LN) on different datasets (ImageNet-C, ImageNet-R), Here, we also conduct sensitivity analysis of in Table 15, which SAR works well under the range [0.03, 0.1].
Table 15: Sensitivity of p in SAR. We report Accuracy (%) on ImageNet-C (shot noise, severity level 5) under online imbalanced label distribution shifts, where the imbalance ratio is xx.
Method
No adapt
Tent
SAR (p=0.01)
SAR (p=0.03)
SAR (p=0.05)
SAR (p=0.07)
SAR (p=0.1)
Accuracy (%)
6.7
1.4
40.3
47.7
47.6
47.2
47.2
25

---

Published as a conference paper at ICLR 2023
G ADDITIONAL DISCUSSIONS
G.1 1 MORE DISCUSSIONS ON MODEL COLLAPSE
As we mentioned in Section 3.1, models online optimized by entropy (Wang et al., 2021; Niu et al., 2022a) under wild test scenarios are easy to collapse, i.e., predict all samples to single class inde- pendent of the inputs. To alleviate this, prior methods (Liang et al., 2020; Mummadi et al., 2021) exploit diversity regularization to force the output distribution of samples to be uniform. However, this assumption is unreasonable at test time such as when test data are imbalanced during period (as in Figure 1 (c)) and this method also relies on a batch of samples (in contrast to Figure 1 (b)). In this sense, this strategy is infeasible for our problem. In paper, resolve the collapse is- sue for test-time adaptation from an optimization perspective to make the online adaptation process stabilized.
G.2 EFFECTS OF LARGE BATCH SIZES IN BN MODELS UNDER MIX DOMAIN SHIFTS
In Section 1, we mentioned that a standard batch size (e.g., 64 on ImageNet) works well when there is only one type of distribution shift. However, when test data contains multiple shifts, this batch size fails to calculate an accurate mean and variance estimation in batch normalization lay- ers. Here, we investigate the effects of super large batch sizes in this setting. From Table 16, the adapted performance increases as the batch size increases, indicating that a larger batch size helps to estimate statistics more accurately. It is worth noting that the performance degrades when BS is larger than 1024. This is because we fix the learning rate for various batch sizes and in this sense, BS=1024 may lead to insufficient model updates. Moreover, although en- larging batch sizes is able to boost the performance, the adapt performance is still inferior to the average accuracy of adapting on each corruption type separately (i.e., average adapt). This further emphasizes the necessity of exploiting models with group or layer norm layers to perform test-time entropy minimization.
severity level 3
Table 16: Effects of large batch sizes (BS) in Tent (Wang et al., 2021) with ResNet-50-Bi under MIXTURE OF DIFFERENT CORRUPTION TYPES on ImageNet-C. We report accuracy (%).
avg. adapt
mix adapt BS=256
Severity
Base
BS=64
BS=32
BS=64
BS=128
BS=512
BS=1,024
Level =5
18.0
42.6
0.9
2.3
4.0
8.3
12.4
16.4
Level 3 39.8
59.0
20.1
40.7
46.7
49.1
49.0
47.8
G.3 PERFORMANCE OF TENT WITH CONVNEXT-LN
For mainstream neural network models, the normalization layers are often coupled with network architecture. Specifically, group norm (GN) and batch norm (BN) are often combined with conven- tional networks, while layer norm (LN) is more suitable for transformer networks. Therefore, we in- vestigate the layer normalization effects TTA in Section through VitBase-LN. Here, we conduct more experiments to compare the performance of online entropy minimization (Wang et al., 2021) 1) on ResNet50-BN and ConvNeXt-LN (Liu et al., 2022). Co a convolutional network equipped with LN. The authors conduct significant modifications over ResNet to make this LN-based convo- lutional network work well, such as modifying the architecture (ResNet block to ConvNeXt block, activation functions, etc.), various training strategies (stochastic depth, random erasing, EMA, etc.). From Table 17, Tent-ConyNeet-LNN performs more stable than Tent+ Reette-eNeetO-BB but still suffers several failure cases. These results consistent with that of ResNet50-Bi BB VitBase-LN.
G.4 EFFECTIVENESS OF MODEL RECOVERY SCHEME WITH TENT AND EATA
In this subsection, we apply our Model Recovery scheme to Tent (Wang et al., 2021) 1) and EATA (Niu et al., 2022a). From Table 18, the model recovery indeed helps Tent lot (e.g., the average accuracy from 22.0% to 26.1% 1% on ResNet50-GNN while its performance gain on EATA is a bit marginal. Com- pared with Tent+recovery and EATA+recovery, our SAR greatly boosts the adaptation performance,
26

---

Published as a conference paper at ICLR 2023
Table 17: Results of Tent on ResNec50-BN and ConvNeXt-LN. We report Accuracy (%) on
ImageNet-C under online imbalanced label distribution shifts and the imbalance ratio is oo.
Blur
Digiul Elastic
Severity level
Gauss.
Shot
Impul.
Defoc.
Glass
Motion
Zoomn
Snow
Frest
Fog
Bri.
Contr:
Pixel
IPEG
Arg.
ResNer50-BN
2.2
2.9
18
17.8
988
14.5
22.5
16.8
23.4
24.6
59.0
17.1
20.7
31.6
18.0
Tore (Wang al., 2021)
1.2
144
1.4
1.0
0.9
12
26
1.7
1.8
3.6
5.0
0.5
2.6
3.2
3.1
2.1
Con-NoXi-LN
Tenn (Wang et 2021)
26.3
11.9
36,9
21.1
127
14.6
5.1
7.8
5.3
6.6
79.0
67.6
08.4
65.8
29.4
Severity level
Gauss.
Shot
Impal.
Defoc.
Glass
Motion
Zoom
Snow
Frost
Fog
Bri.
Contr:
Blastic
Pixel
JPBG
Arg.
ResNet50-BN
27.7
25.2
25.1
37.8
16.7
57.8
35.3
35.2
32.1
46.7
69.5
46.2
55.4
46.3
59.4
39
Tork (Wang at. 2021)
34
3.2
3.2
2.3
2.0
2.4
14
244
2.4
4.6
544
3.0
4.8
4.6
4.1
3,4
Con-NeXi-LN
Tenr (Wang et 2021)
73.4
73.5
74.4
06.8
61.6
71.4
64.2
22.6
39.5
76.8
81.2
78.9
76.8
75.6
75.8
67.5
e.g., the average accuracy 26. 1% (Tent+recovery) vs. 37.2% (SAR) ResNet50-GN, suggesting the effectiveness of our proposed SAR.
Table 18: Results of combining model recovery scheme with Tent and EATA. We report the Accu- racy (%) on ImageNet-C severity level under online imbalanced label distribution shifts and the imbalance ratio is co.
Noisc Shot
Blur Matiinn
Weather
Digital
ModelsMMttiod
Gauss,
Impul.
Defoee
Fog
Bril:
Contr:
Hlastic
Pixel
JPSs
ResNer50 (GN)
18.0
19.8
21,4
40.4
47.3
69.3
28.4
Tent
2.6
3.3
2.7
13.9
7.9
19.5
17.0
16.5
21.9
1.8
70.5
42.2
6.6
49.4
53.7
22.0
Tentvrecover
12.2
EATA
27.0
28.3
28.1
14.9
17.1
24.4
25.3
32.2
32.0
39.8
66.7
33.6
24.5
41.9
38.4
31.6
SAR (ours)
33.1
36.5
35.5
19.2
19.5
33.3
21.7
23.9
45.3
50.1
71.9
46.7
7.1
52.1
56.3
37.2
VitRase (I.N)
9.4
6.7
8.3
29.1
23.4
34.0
27.0
15.8
26.3
47.4
$4.7
43.9
30.5
44.5
47.6
29.9
1.4
TenL+rrcover
40.3
10.1
42.4
$4.4
52.3
58.1
52.2
31.6
39.2
69.3
76.1
66.1
56.7
69.4
66.4
52.3
FATA-reawwer
35.9
34.6
36.7
45.3
47.2
49.3
41.7
56.5
55.4
62.2
72.2
21.7
56.2
64.7
63.7
49.9
SAR (ours)
46.5
43.1
48.9
55.3
34.3
58.9
54.8
$3.6
46.2
69.7
76.2
66.2
60.9
69.6
66.6
58.1
27


Reviews:
[{"review_id": "yl7yqtp_etT", "reviewer": "Reviewer_7Sq7", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper investigates the pitfalls of online domain adaptation, that the model tends to collapse with small batch sizes and class imbalances at test time. The authors also propose two techniques to mitigate the problems. ", "summary_of_the_review": "Below are my concerns, which I hope the authors will address:\n**The two solutions proposed by the authors do not seem new to me**:\n- The entropy selective term $S(x)$ looks a lot like the one from EATA. In fact, it is almost the same thing, just without the coefficient term. It is also worrying to me that the authors do not mention nor discuss this similarity.\n- The sharpness-aware minimization technique is also not new. It is adopted directly from SAM from the supervised loss to the self-supervised loss. On a side note, the authors mentioned a problem with gradient selection/clipping is that the threshold needs careful tuning. However, I would also argue that the hyper-parameter $\\rho$ for the sharpness-aware entropy minimization also needs careful tuning, since it should depend on the norm of the parameters of the source-pretrained model.\n\n**The empirical evaluation seems incomprehensive**\n- The authors only validate their method for the case of entropy minimization. To the best of my knowledge, there are more surrogate losses that are often used for ODA. It would be beneficial to test if the method also helps in these cases.\n- I wonder why the authors do not test their method with BN models (e.g., original resnet50)? If it does not perform well in this case, I suggest adding some explanation/discussion regarding this.\n- It would also be beneficial to conduct experiments on domain shift datasets such as VisDA.\n\n\n===========================\nPost rebuttal:\n\nI would like to thank the authors for their detailed response and the effort they put into this rebuttal.\n\nThey addressed most of my concerns, but some points remained (detailed below). For this reason, I increase my score from 3 to 5, since I think some arguments still need to be sharpened in the final draft.\n\n- In the comparison with EATA and SAM, the authors pointed out some differences between their proposed method and these existing works (mostly about the purposes of the work, not much about the methodologies). With such similarities in the methodologies of the methods, it is absolutely important to acknowledge and discuss these details in the paper. As far as I am aware, the authors did not include any new discussions regarding this in the revision. Note that it is okay to (borrow)/(take motivation from) existing works, but discussion about the similarities, differences and novelties are needed for completeness.\n\n- The authors mentioned they do not apply their method to other test time adaptation method, such as TTT, TTT+,... since they are computationally expensive (require several forward/backward passes). I find this arguement not convincing. If some methods are more expensive but offer better performance, they are still worth some value (some people even prefer these methods more for the performance). As the paper is deadling with TTA in the wild, I think it is important to cover a wide range of settings, including the TTA method.", "strength_and_weaknesses": "**Strengh**\nThe paper is easy to follow. Authors also include explanations/intuitions to certain phenomena and design choices.\n\n**Weakness:** Follows are my concerns. More detail below.\n- The proposed method somewhat lacks novelty.\n- The empirical evaluation can be improved and strengthened.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "gb_-T55DdL", "reviewer": "Reviewer_AHGC", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose a test-time adaptation (TTA) method to deal with distribution shifts between training and test data. Their method aims to deal with the unstable issue of online model updating in TTA. A sharpness-aware and reliable entropy minimization method is designed. They also remove partial noisy samples and encourage model weights to go to a flat minimum to further stabilize TTA.", "summary_of_the_review": "Please address concerns mentioned in weaknesses.", "strength_and_weaknesses": "* Strengths:\n    \n    1. This work provides detailed investigation of unstable reasons and careful analysis of failure cases in TTA. According to their investigation and analysis, a sharpness-aware and reliable method is proposed to solve those issues.\n\n    2. Their discussion about norm layer is interesting. They also find weaknesses of only using GN and LN and provide a good solution to deal with such issues. The analysis of test sample gradients and the corresponding removing method is quite novel and efficient.\n\n    3. Their experiments and ablation studies demonstrate that their analysis and hypothesis is indeed true, and the method they proposed is effective to improve the performance of TTA on ImageNet-C.\n\n* Weaknesses:\n\n    1. Their analysis about the norm layer is mainly based on empirical evidences. It is good to support their study with comprehensive experiments. However, I still wonder is there any intuitive explanation about choosing GN and LN instead of BN? This could help readers to better understand the analysis.\n\n    2. In Fig. 6, the authors find even with fine-tuned hyper-parameters, gradient clipping yield worse results than no clipping, which is a bit counter-intuitive. Is there any explanation about the failure of gradient clipping even with fine-tuning?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "XcWl7CzGgXi", "reviewer": "Reviewer_Ww1T", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper focuses on fully test-time adaptation in dynamic and \"wild\" settings, in which (i) data from different target domains might be mixed at test time, (ii) test samples might not come in batches, and (iii) test samples might not be independently sampled in the sense that class frequency p(y) differs across batched. The authors identify batch norm as being harmful under such a setting and identify networks using group or layer norm to be preferable. Moreover, the authors propose sharpness-aware and reliable entropy minimization (SAR), to prevent model collapse caused by samples with large gradient norms. SAR consists of using the sharpness aware minimizer (SAM), entropy-based filtering of test samples used for adaptation, and a heuristic model recovery scheme. An empirical study in the dynamic \"wild\" settings and the proposed approaches is conducted on domain shifts with common corruptions.\n", "summary_of_the_review": "Overall, I think this paper is a solid contribution that deserves acceptance at ICLR. The main contribution for me is to study and extend TTA in a more realistic \"wild\" setting, which should increase practical relevancy of TTA. The methods themselves are not novel but their combination and fit to \"wild\" TTA are sufficiently novel.", "strength_and_weaknesses": "Strengths:\n * the proposed setting of \"wild\" test-time adaptation is of high practical relevancy and it is considerably more challenging that the standard setting of batched, single-target domain, class-balanced TTA. The authors also motivate and illustrate this setting well (compare Figure 1)\n * the paper does a good job in empirically identifying problematic parts of existing TTA approaches in the \"wild\" setting, namely (i) batch-norm's implicit single-domain assumption and reliance on large batches and (ii) model collapse caused by large gradient norms. \n * Pragmatic solutions to these issues are identified that are not inherently novel but shown to be very efficient in the \"wild\" setting: (i) using group/layer norm instead of batch-norm and (ii) using SAR (SAM + entropy-based filtering + model recovery) to prevent model collapse.\n * Overall, the paper is strong on the empirical side and provides an extensive set of experiments on the different parts of the \"wild\" setting and the different components of the proposed wild TTA procedure. \n * It is laudable that runtimes of different TTA-procedures are compared in Table 1. I would recommend to use the same unit (seconds) for all methods though. One question: why has SAR nearly the same runtime as TENT? I would expect that the double backward pass causes it to be somewhat slower. \n\nWeaknesses:\n * empirical evaluation is limited to target domains with common corruptions (e.g. ImageNet-C); investigation of other types of domain shifts like ImageNet-R would provide additional insights in terms of the generality of the proposed procedures.\n * the paper's comparison of normalization layers has a strong confounding factor in the overall model architecture: while batch-norm and group-norm are studied as part of a ResNet-50, layer-norm is studied as part of an ViT. Thus, a statement like \"GN models are more suitable for self-supervised TTT than LN models since TTT+VitBase-LN is sensitive to different sample orders and has large variances over different runs.\" is problematic since it might not be that differences are due to the normalization layer (but rather to the model architectures). It would be preferable to remove the confounding factor of model architecture in this part.\n * The ablation study of SAR in Table 5 is nice but should be extended to cover all components of the wild setting and not only imbalanced labels\n * the model recovery scheme seems to be very helpful in the ViT-LN setting. To which extent would it also benefit TENT and EATA? That is: to which extent are benefits of SAR over TENT/EATA due to model recovery and not due to SAR?\n * not really a weakness, but a natural question arising would be: how would a model trained with SAM on the source domain perform in the comparisons given that SAM was found to be helpful at test-time?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "FrmxbyaX-n", "reviewer": "Reviewer_w35R", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new method named sharpness-aware and reliable entropy minimization (SAR) to stabilize wild TTA. Specifically, to mitigate the collapse of group norm or layer norm models, SAR first filters samples with high and noisy gradients according to entropy loss. Furthermore, SAR introduces a sharpness-aware learning scheme to ensure that model weights remain at a flat minimum during optimization. Extensive experiments demonstrate the importance of Wild TTA and the effectiveness of SAR.", "summary_of_the_review": "This paper is easy to read and novel but still has some problems. Please refer to the weakness.", "strength_and_weaknesses": "Strength:\n1. The authors introduce wild online TTA settings. For example, the test data may have 1) mixture distribution variation, 2) mini-batch, 3) online imbalanced label distribution variation, which are realistic during application.\n\n2. Extensive experiments are conducted to illustrate the limitations of Batch Norm in TTA and further observed that batch-independent norms (e.g., Group Norm and Layer Norm) will be more stable than Batch Norm. The experiments provide new insights for the TTA community.\n\n3. The authors observe that models with Group Norm or Layer Norm tend to collapse, especially when the severity of the distribution shift is high. They then propose a sharpness-aware learning scheme to alleviate the collapse problem, which is technically reasonable and novel within the scope of TTA.\n\n4. The authors propose a sharpness-aware and reliable entropy minimization method to uniformly address the instability of previous methods under three wild TTA settings. Extensive experiments on ImageNet-C demonstrate the effectiveness of SAR.\n\nWeakness:\n1. This paper focuses on dynamic wild-test time adaptation settings. I am curious about under what circumstances does the third situation arise? Can the author provide some examples?\n\n2. In Tables 2 and 4, all considered methods (including the proposed method) achieve worse TTA performance than baselines on ResNet50-GN with damaged Defoc. That is, all SOTA TTA methods cannot fit out-of-distribution samples. Could the authors discuss more about this phenomenon?\n\n3. In Section 4, the description is a bit difficult to understand. Since these figures are informative, it is best to indicate which subgraph is being described when describing.\n\n4. The numbers in section 4 are somewhat difficult to understand directly and the font size is too small. It's best to optimize these numbers for readability.\n\n5. In Table 2, were the results for different damages measured with the same sample sequence?\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-5%