Test-Time Adaptation to Distribution Shifts by Confidence Maximization and Input Transformation
Authors: Chaithanya Kumar Mummadi
Robin Hutmacher
Kilian Rambach
Evgeny Levinkov
Thomas Brox
Jan Hendrik Metzen
Venue: ICLR 2022 Submitted (2022)
Keywords: natural corruptions
corruption robustness
distribution shift
test time adaptation
domain shift

Abstract: Deep neural networks often exhibit poor performance on data that is unlikely under the train-time data distribution, for instance data affected by corruptions. Previous works demonstrate that test-time adaptation to data shift, for instance using entropy minimization, effectively improves performance on such shifted distributions. This paper focuses on the fully test-time adaptation setting, where only unlabeled data from the target distribution is required. This allows adapting arbitrary pretrained networks. Specifically, we propose a novel loss that improves test-time adaptation by addressing both premature convergence and instability of entropy minimization. This is achieved by replacing the entropy by a non-saturating surrogate and adding a diversity regularizer based on batch-wise entropy maximization that prevents convergence to trivial collapsed solutions. Moreover, we propose to prepend an input transformation module to the network that can partially undo test-time distribution shifts. Surprisingly, this preprocessing can be learned solely using the fully test-time adaptation loss in an end-to-end fashion without any target domain labels or source domain data. We show that our approach outperforms previous work in improving the robustness of publicly available pretrained image classifiers to common corruptions on such challenging benchmarks as ImageNet-C.

Under review as a conference paper at ICLR 2022
TEST-TIME ADAPTATION TO DISTRIBUTION SHIFT BY CONFIDENCE MAXIMIZATION AND INPUT TRANSFOR- MATION
Anonymous authors Paper under double-blind review
ABSTRACT
Deep neural networks often exhibit poor performance on data that is unlikely under the train-time data distribution, for instance data affected by corruptions. Previous works demonstrate that test-time adaptation to data shift, for instance using entropy minimization, effectively improves performance on such shifted distributions. This paper focuses on the fully test-time adaptation setting, where only unlabeled data from the target distribution is required. This allows adapting arbitrary pretrained networks. Specifically, we propose novel loss that improves test-time adaptation by addressing both premature convergence and instability of entropy minimization. This achieved by replacing the entropy by non-saturating surrogate and adding a diversity regularizer based on batch-wise entropy maximization that prevents convergence to trivial collapsed solutions. Moreover, we propose to prepend an input transformation module to the network that can partially undo test-time distribution shifts. Surprisingly. this preprocessing can be learned solely using the fully test-time adaptation loss in an end-to-end fashion without any target domain labels or source domain data. We show that our approach outperforms previous work in improving the robustness of publicly available pretrained image classifiers to common corruptions on such challenging benchmarks as ImageNet-C.
INTRODUCTION
Deep neural networks achieve impressive performance on test data, which has the same distribution as the training data. Nevertheless, they often exhibit a large performance drop on test (target) data which differs from training (source) data; this effect is known as data shift Anrionero-Cander et al.) 2009) and can be caused for instance by image corruptions. There exist different methods to improve the robustness of the model during training (Geirhos et al., 2019: Hendrycks et al. 2019 Tzeng et al., 2017). However, generalization to different data shifts limited since is infeasible to include sufficiently many augmentations during training cover the excessively wide range of potential data shifts (Mintun et al. 2021a). Alternatively, in order to generalize to the data shift at hand, the model be adapted during test-time. Unsupervised domain adaptation methods such as Vu et al. (2019) use both source and target data to improve the model performance during test-time. In general source data might not be available during inference time, e.g., due to legal constraints (privacy or profit). Therefore we focus on the fully test-time adaptation setting (Wang et al. 2020 model is adapted to the target data during test time given only the arbitrarily pretrained model parameters and unlabeled target data that share the same label space as source data. We extend the work of Wang et al. (2020) by introducing novel loss function, using diversity regularizer, and prepending a parametrized input transformation module to the network. We show that our approach outperform previous works and make pretrained models robust against common corruptions on image classification benchmarks as ImageNet-C (Hendrycks & Dietterich 2019) and ImageNet-R (Hendrycks et al. 2020).
Sun et al. (2020) investigate test-time adaptation using self-supervision task. Wang et al. 2020) and Liang et al. 22020) use the entropy minimization loss that uses maximization of prediction confidence as self-supervision signal during test-time adaptation. Wang et al. (2020) has shown that such loss performs better adaptation than a proxy task (Sun et al. 2020). When using entropy minimization, however, high confidence predictions do not contribute to the loss significantly anymore and thus provide little self-supervision. This is drawback since high-confidence samples provide the most

---

Under review as a conference paper at ICLR 2022
trustworthy self-supervision We mitigate this by introducing two novel loss functions that ensure that gradients of samples with high confidence predictions do not vanish and learning based on self-supervision from these samples continues. Our losses do not focus minimizing entropy but on minimizing the negative log likelihood ratio between classes; the two variants differ in using either soft or hard bseudo-labels, In contrast entropy minimization, the proposed loss functions provide non-saturating gradients, even when there are high confident predictions, Figure provides illustration of the losses and the resulting gradients. Using these new loss functions, we are able to improve the network performance under data shifts in both online and offline adaptation settings.
In general, self-supervision by confidence maximization can lead to collapsed trivial solutions, which make the network to predict only a single or a set of classes independent of the input. To overcome this issue diversity regularizer (Liang et al., 2020 Wu et al. 2020) can be used, that acts on a batch of samples. It encourages the network to make diverse class predictions on different samples. We extend the regularizer by including a moving average, in order to include the history of the previous batches and show that this stabilizes the adaptation of the network to unlabeled test samples. Furthermore we also introduce a parametrized input transformation module, which we prepend to the network. The module is trained in fully test-time adaptation manner using the proposed loss function, and without using source data or target labels. It aims to partially undo the data shift at hand and helps to further improve the performance on image classification benchmark with corruptions.
Since our method does not change the training process, it allows to use any pretrained models. This beneficial because any good performing pretrained network can be readily reused, e.g., network trained on some proprictary data not available to the public. We show, that our method significantly improves performance of different pretrained models that are trained on clean ImageNet data.
In summary our main contributions are as follows: we propose non-saturating losses based on the negative log likelihood ratio, such that gradients from high confidence predictions still contribute to test-time adaptation. We extend diversity regularizer to its moving average to include the history of previous batch samples to prevent the model collapsing trivial solutions. We also introduce an input transformation module, which partially undoes the data shift at hand. We show that the performance of different pretrained models can be significantly improved on ImageNet-C and ImageNet-R.
2 RELATED WORK
Common image corruptions are potentially stochastic image transformations motivated by real- world effects that can be used for evaluating a model's robustness. One such benchmark, ImageNet- C. (Hendrycks & Dietterich, 2019), contains simulated corruptions such as noise, blur, weather effects, and digital image transformations. Additionally, Hendrycks et al. (220)) proposed three data sets containing real-world distribution shifts, including Imagenet-R. Most proposals for improving robustness involve special training protocols, requiring time and additional resources. This includes data augmentation like Gaussian noise (Ford et al., [219 Lopes et al. 2019; Hendrycks et al.) 2020), CutMix (Yun et al.||2119) 2019), AugMix (Hendrycks et al.) 2019). training on stylized images (Geirhos et al., 2019; Kamann et al. (020) or against adversarial noise distributions (Rusak et al.) 2020a). Mintun et al. (2021b) pointed out that many improvements on ImageNet-C are due to data augmentations which are too similar to the test corruptions, that is: overfitting to ImageNet-CC occurs. Thus, the model might be less robust to corruptions not included in the test set of ImageNet-C.
Unsupervised domain adaptation methods train a joint model of source and target domain by cross- domain losses to find more general and robust features, e. g. optimize feature alignment (Quiñonero Candela et al.) 2008 Sun al. 2017) between domains, adversarial invariance (Ganin Lempitsky 2015; Tzeng et al.) 2017 Ganin et al. 2016 Hoffman et al.] 2018). shared proxy tasks (Sun et al. 2019) or adapt entropy minimization via an adversarial loss (Vu et al. 2019). While these approaches effective, they require explicit access to source and target data the same time, which may not always be feasible. Our approach works with any pretrained model and only needs target data.
Test-time adaptation is setting, when training (source) data is unavailable test-time. It is related to source free adaptation, where several works use generative models, alter training (Kundu et al.
2020 Li al. [2020b] Kurmi et al. [222 Yeh et al. (021)) and require several thousand epochs to 2021) Besides, there is another line of work
adapt to the target data (Li et al.) 2020b; Ych et al.
(Sun al. 2020 Schneider et al. 2020 Nado al.
2021 Benz al. 2021 Wang al. 2020) that
2

---

Under review as a conference paper at ICLR 2022
interpret the common corruptions as data shift and aim improve the model robustness against these corruptions with efficient test-time adaptation strategy to facilitate online adaptation. such settings spare the cost of additional computational overhead. Our work also falls in this line of research and aims to adapt the model to common corruptions efficiently with both online and offline adaptation.
Sun et al. (2020) update feature extractor parameters at test-time via self- proxy task (predicting image rotations). However, [Sun et al. (2020) alter the training procedure by including the proxy loss into the optimization objective as well, hence arbitrary pretrained models cannot be used directly for test-time adaptation. Inspired by the domain adaptation strategies (Maria Carlucci
ct al. 2017 Li et al., 2016), several works (Schneider et al., 2020 Nado al. 2021] Benz et al. 2021 replace the estimates of Batch Normalization (BN) activation statistics with the statistics of the corrupted test images. Fully test time adaptation, studied by Wang al. (2020) (TENT) uses entropy minimization to update the channel-wise affine parameters of BN layers on corrupted data along with the batch statistics estimates. SHOT (Liang et al.. 2020) also uses entropy minimization and a diversity regularizer to avoid collapsed solutions. SHOT modifies the model from the standard setting by adopting weight normalization at the fully connected classifier layer during training to facilitate their pseudo labeling technique. Hence, SHOT is not readily applicable to arbitrary pretrained models.
We show that pure entropy minimization Wang et al. 2020 Liang et al., 2020) as well as alternatives such max square loss (Chen et al. 2019) and Charbonnier penalty (Yang & Soatto 2020) results in vanishing gradients for high confidence predictions, thus inhibiting learning. Our work addresses this issue by proposing a novel non-saturating loss, that provides non-vanishing gradients for high confidence predictions. We show that our proposed loss function improves the network performance through test-time adaptation. In particular, performance on corruptions of higher severity improves significantly. Furthermore, we add and extend the diversity regularizer (Liang et al. 2020; Wu et al., 2020) to avoid collapse to trivial, high confidence solutions. Existing diversity regularizers (Liang et al., 2020 Wu et al., (020) act on batch of samples, hence the number of classes has to be smaller than the batch size. We mitigate this problem by extending the regularizer to moving average version. Li et al. (020a) also use a moving average to estimate the entropy of the unconditional class distribution but source data is used to estimate the gradient of the entropy. In contrast, our work does not need access to the source data since the gradient is estimated using only target data. Prior work Tzeng et al. (2017) Rusak et al. (2020b); Talebi & Milanfar (202T) transformed inputs by an additional module overcome domain shift, obtain robust models, and also to learn resize. In our work, we prepend an input transformation module to the model, but in contrast to former works, this
module is trained purely at test-time to partially undo the data shift hand to aid the adaptation.
METHOD
We propose a novel method for fully test-time adaption. We assume that a neural network fo with parameters 0 is available that was trained on data from some distribution D, as well set of (unlabeled) samples X ~ D' from a target distribution D' * 'D (importantly, no samples from DD are required). We frame fully test-time adaption as two-step process: (i) Generate novel network gø based on fo-
where & denotes the parameters that are adapted. A simple variant for this is y f and o C 0 Wang let al. (2020). However, we propose more expressive and flexible variant in Section 3. (ii) Adapt the parameters & of on X using an unsupervised loss function L. We propose two novel losses Letr and Lnir in Section 3.2 that have non-vanishing gradients for high-confidence self-supervision.
3.1 INPUT TRANSFORMATION
We propose to define the adaptable model as g= f d. That is: we preprend a trainable network d to f. The motivation for the additional component d is to increase expressivity of such that can learn to (partially) undo the domain shift DD -> D'. Specifically, we choose d(x)) = 7 [rz + - (T-r)r(())) + B, where r E IR, (3, 7) R with nin being the number of input channels, Ty being a network with identical input and output shape, and denoting elementwise multiplication. Specifically, ,8 and 7 implement channel-wise affine transformation and r implements convex combination of unchanged input and the transformed input (z). By choosing T = 1,, =, 1, B = O, ensure d(x: x and thus g at initialization. In principle, T' can be chosen arbitrarily. Here, we choose rgg as a simple stack of 3 x 3 convolutions, group normalization, and ReLUs (refer Sec. A.2 for details). However, exploring other choices would be an interesting avenue for future work.
3

---

Under review as a conference paper at ICLR 2022
Importantly, while the motivation for is to learn to partially undo a domain shift D + D', we train end-to-end in the fully test-time adaptation setting on data X ~ D', without any access to samples from the source domain D, based the losses proposed in Section 3.2 The modulation parameters of 9o are Ø = (3, y; t/, 0²), where 0' C 0. That is, we adapt only a subset of the parameters 0 of the pretrained network We largely follow Wang et al. (2020) in adapting only the affine parameters of normalization layers in f while keeping parameters of convolutional kernels unchanged. Additionally, batch normalization statistics (if any) are adapted to the target distribution.
Note that the proposed method is applicable to any pretrained network that contains normalization layers with a chamnel-wise affine transformation. For networks with no affine transformation layers, one can add such layers into f that are initialized to identity as part of model augmentation.
3.2 ADAPTATION OBJECTIVE
We propose a loss function I= Ldiv + 5eont for fully test-time network adaptation that consists of two components: (i) a term Laiv that encourages predictions of the network over the adaptation dataset X that match target distribution pD (y). This can help avoiding test-time adaptation collapsing to too narrow distributions such as always predicting the same or very few classes. If pr> pD>(y) is (close to) uniform, it acts diversity regularizer. (ii) A term Leaat that encourages high confidence prediction on individual datapoints. We note that test-time entropy minimization (TENT) (Wang et al., 2020) fits into this framework by choosing Laiv = 0 and Lcont the entropy.
3.2.1 CLASS DISTRIBUTION MATCHING Ldiv
Assuming knowledge of the class distribution PD (y) on the target domain D', we propose to add a term to the loss that encourages the empirical distribution of (soft) predictions of go on X to match this distribution. Specifically, let POo (y) be an estimate of the distribution of (soft) predictions of Go. We use the Kullback-Leibler divergence Laiv = (f (y)ll p>> (y)) as loss term. In some applications information about the target class distribution is available, e.g. in medical data it might be known that there is a large class imbalance. In general this information is not available, and here we assume uniform distribution of PD (p) which corresponds to maximizing the entropy H (fge (v)). Similar assumption has been made in SHOT to circumvent the collapsed solutions.
Since the estimate P.. (y) depends on ø. which is continuously adapted, it needs to be re-estimated on a per-batch level. Since re-estimating Poo (y) from scratch would be computational expensive, we propose use running estimate that tracks the changes of & as follows: let Pt-1 (y) be the estimate at iteration t 1 and -.mpp
2"=1g:1), where ((*)) are the predictions (confidences) of go mini-batch of n inputs (*) ~ X. We update the running estimate via pe (y) = K sg(p:-1(y)) + (1 k)) Pr pmP,, where sg refers stop- noo-ggradeet. The loss becomes Laiv = D z (Pr (v)ll P (v)) accordingly. Unlike Li et al. (2020a), our approach only requires target but no source data to estimate the gradient.
3.2.2 CONFIDENCE MAXIMIZATION Leconf
We motivate our choice of Lcont step-by-step from the (unavailable) supervised cross- loss: for this, let ğ = (z) be the predictions (confidences) of model go and H (ū.y") y²) = E. ve log 9e be the cross-entropy between prediction ğ and some reference y". Let the last layer of be a softmax activation layer softmax. That ý = softmax(o), where o are the network's logits. We can rewrite the cross-entropy in terms of the logits o and one-hot reference y" as follows: H (softmax(()) v") -00p- + log 2 eo where cº is the index of the I in y" and ne is the number of classes.
When labels being available for the target domain (which we do not assume) in the form of one-hot encoded reference y/ for data Ft, one could use the supervised cross-entropy loss by setting y" = y/ and using Leup (ÿ. 5²) H (ū. v²) H (ū, ys). Since fully test-time adaptation assumes no label information, supervised cross-entropy loss is not applicable and other options for y" need to be used.
One option is (hard) pseudo-labels. That is, one defines the reference y" based on the network predic- tions z via y" = onchoo(t) where onchot creates a one-hot reference with the I corresponding to the class with maximal confidence in ÿ. This results in Ly c² = arg max ğ. One disadvantage with this loss is that the (hard) pseudo-labels ignore uncertainty in the network predictions during self-supervision. This results in large gradient magnitudes with
H (ô. onehot())) = - log De-, with
4

---

Under review as a conference paper at ICLR 2022
0.0
00.2
Gradiett au
80y
-0.4
-0.6
Loss L
Hard Likelhosd Ratio
-0.2
-0.8
Soft LBedisood Radi
Entray Hard Pauuddabel
-9.4
-1.0
0.4
0.6
0.2
0.4
1.0
Confidence
Confidence yn
Figure 1: Illustration of different losses for confidence maximization. Losses (left, shifted such that maxima of all losses are at 0) and the resulting gradients with respect to the first logit (right) as function of the first classes confidence are shown for the case of a binary classification problem. Both entropy and hard pseudo-labels have vanishing gradients for high confidence predictions" Accordingly, both have maximum gradient amplitude for low-confidence self-supervision, with this effect being stronger for the hard pseudo-labels. Hard Likelihood Ratio has constant gradient amplitude for any confidence and thus takes into account low- and high-confidence self-supervision equally. Soft Likelihood Ratio also shows non-vanishing (albeit non-maximum) gradients for high- confidence self-supervision and additionally produces small gradient amplitudes from low-confi confidencc self-supervision. Since the likelihood ratio-based losses unbounded, the design of the model needs to ensure that logits cannot grow unbounded. 'ResNet50 has median prediction confidence of 0.929 on the ImageNet validation data and hence we consider confidence of < 0.8 as low and 2 0.95 as high.
respect the logits Dhpt being generated on data where the network has low confidence (see Figure Bo.-
[]. This is undesirable since it corresponds the network being affected most by data points where the network's 's self-supervision is least reliabld"]
An alternative to use soft pscudo-labels, that is 3" ÿ. This takes uncertainty in network predictions into account during self- self-labelling and results in the entropy minimization loss of TENT (Wang et al. 2020): Lent(() H(. D) H())
E. Ue log Dc. However, also for the entropy the logits' gradient magnitude 2 goes to O when one of the entries in á goes to I (see Figure Dh. For binary classification task, for instance, the maximal logits' gradient amplitude is obtained for ÿ R (0.82, 0.18). This implies that during later stages of test-time adaptation where many predictions typically already have high confidence (significantly above 0.82), gradients are dominated by datapoints with relative low confidence in self-supervision.
While both hard and soft pseudo-labels are clearly motivated, they are not optimal in conjunction with a gradient-based optimizer since the self-supervision from low confidence predictions dominates (at least during later stages of training). We address this issue by proposing two losses that increase the gradient amplitude from high confidence predictions. We argue that this leads to stronger self- supervision (better gradient direction when averaged over the batch) than from the entropy loss (see also Sec. [ for an illustrative example supporting this claim). The two losses are analogous to Lp and Lent. but are not based on the cross-entropy I/ but on the negative log likelihood ratios:
R(D.v) -v.. log
Je
Σ (log de log Σ -+)= H().v") H(A
Σ Uo log Σe U:
Eito
Note that while the entropy H is lower bounded by O, R can get arbitrary small if vT + 1 and the sum Eite 0; + 0 and thus log Eife 91 -o. This property will induce non-vanishing gradients for high confidence predictions.
The first loss we consider is the hard likelihood ratio loss that is defined similarly to the hard pseudo-labels loss Lpt:
de- = ---loge log( Eiter ea, -0. + log Σ c
eo
Lutr (0) R(ů, onchot(0)) = - -log( Exjcr 01
'The prediction confidence for a datapoint can be interpreted as a proxy for its distance to the decision boundary. A low confidence prediction indicates that a datapoint appears to be close to the decision boundary and the model is less certain on which side of the decision boundary the datapoint should lie. We call this "low confidence self-superrisiion since the direction of the gradient becomes ambiguous.
5

---

Under review as a conference paper at ICLR 2022
where c* arg mag ÿ. We note that Shate = 1, thus also high-confidence self-supervision dog
contributes equally to the maximum logits' gradients. This loss also independently proposed as negative log likelihood ratio loss by Yao et al. (2020) as a replacement to the fully-supervised cross entropy loss for classification task. However, to the best of our knowledge, we the first to motivate and identify the advantages of this loss for self-supervised learning and test-time adaptation due to its non-saturating gradient property.
In addition to Lite, we also account for uncertainty in network predictions during self-labelligg in a similar way as for the entropy loss Lent, and propose the soft likelihood ratio loss:
Lstr - R(O.ROO 0) = - Σ Dc log(
Dc
Σ Ec(--o - log Σ eth)
Σizo D4
We note that as Gc- + 1, Larr (O). Thus the asymptotic behavior of the two likelihood ratio losses for high confidence predictions is the same. However, the soft likelihood ratio loss creates lower amplitude gradients for low confidence self-supervision. We provide illustrations of the discussed losses and the resulting logits' gradients in Figure [] Furthermore, an illustration of other losses like the max square loss and Charbonnier penalty can be found in Sec. A.7
We note that both likelihood ratio losses would typically encourage the network to simply scale its logits larger and larger, since this would reduce the loss even if the ratios between the logits remain constant. However, when finetuning an existing network and restricting the layers that adapted such that the logits remain approximately scale-normalized, these losses can provide a useful and non-vanishing gradient signal for network adaptation. We achieve this appproximate scale normalization by freezing the top layers of the respective networks. In this case, normalization layers such as batch normalization prohibit "logit explosion". However, predicted confidences can presumably become overconfident; calibrating confidences in a self-supervised test-time adaptation setting is an open and important direction for future work.
EXPERIMENTAL SETTINGS
Datasets We evaluate our method on image classification datasets for corruption robustness and domain adaptation. We evaluate on the challenging benchmark ImageNet-C (Hendrycks & Dietterich 2019), which includes a wide variety of 15 different synthetic corruptions with 5 severity levels that attribute to data shift. This benchmark also includes 4 additional corruptions as validation data. For domain adaptation, we choose ImageNet trained models to adapt to ImageNet-R proposed by Hendrycks et al. (2020). ImageNet-R comprises 30,000 image renditions for 200 ImageNet classes.
Domain adaptation on VisDA-C (Peng et al. 2017) and digit classification can be found in Sec. A.6
Models Our method operates in fully test-time adaptation setting that allows to any arbitrary pretrained model. We use publicly available ImageNet pretrained models ResNet50, DenseNet121, ResNeXt50, MobileNetV2 from torchvision TTorch-Contribun (2020). We also test robust ResNet50 model trained using AueppttunnttAuugu
Hendrycks et al. 2020).
Baseline for fully test-time adaptation Since TENT from Wang et al. (020) outperformed compet- ing methods and fits the fully test-time adaptation setting, we consider it as a baseline and compare results to this approach. Similar to TENT, we also adapt model features by estimating the normal- ization statistics and optimize only the channel-wise affine parameters on the target distribution.
Settings We conduct test-time adaptation on target distribution with both online and offline updates using the Adam optimizer with learning rate 0.0006 with batch size 64. We set the weight of Lconf in our loss function to 8 0.025 and 0.9 in the running estimate pr(y) of Lai (we investigate the effect of in the Sec. A.4). Similar to SHOT (Liang et al. 2020), we also choose the target distribution PD PD-(y) in Lav as a uniform distribution over the available classes. For TENT, we use SGD with momentum 0.9 at learning rate 0.00025 with batch size 64. These values correspond to the ones of Wang et al. (2020); alternative settings for TENT did not improve performance. For offline updates, we adapt the models for 5 epochs using cosine decay schedule of the learning rate. We found that the models converge during 3 to epochs and do not improve further. Similar to Wang ct al. 2020), we also control for ordering by data shuffling and sharing the order across the methods.
Frrm http:://ghhiaabccoo////ccc///ngn
Owner permitted to use it for rescarch/commercial purposes. purpossss

---

Under review as a conference paper at ICLR 2022
Table 1: Test-time adaptation of ResNet50 on ImageNet-C at highest severity level 5. Ground truth labels are used to adapt the model in supervised manner to obtain empirical upper bound performance.
Method
Gauss
Shot
Impulse
Defocus
Gloes
Motion
Zoom
Snow
Frost
Flg
Bright
Costrast
Elasti
Pixel
JPEG
mess
No Adaptation
2.46
2.99
1.96
17.92
9.82
14.78
22.50
16.89
23.
24.43
55, 93
5:43
16.95
20.61
31,65
18.04
Pseudo Labels
2.44
2,99
1.96
17.92
9.82
14.78
22.50
16,89
23.31
24.43
58.93
5.43
16.95
20.01
31.65
18.04
Ouline odaptation (evaluation on botch Gi Cireelly after adagtarion the bacch)
TENT
60
31.06
50.54
29.09
28.07
42.32
50. 31
45.01
42.05
58,40
68.20
27.25
55.68
99.46
53.64
43.51
TENT= 29.09
31.65
30.68
29.3
28.6 65
42.30
50.32
48.09
42.54
39
65.23
31.43
55.90
59,
53.66
43.96
HLR SLR (ours)
33.10
36,05
34.74
33.21
52.77
42-75
58.02
47.40
35,11
37,93
36.83
35.15
35.13
48.29
53.45
52.08
46.52
60.74
68.40
44.78
58.74
61.15
55,97
48.72
Foslustion
opech
TENT
32.44
37.40
$3.09
43.35
24.35
56.53
61.00
33.75
33.43
47.66
33.43
61.75
HLR (ours)
18.25
51.63
62.19
62.51
57.42
SLR (ours)
39.51
41.58
39.35
39,02
$2.67
$5,80
$5.92
49.64
62.62
68.47
50.27
60.80
63.01
57.90
Fvslustion afterr opecchs
TENT
30.64
33.80
34.72
30.13
29.05
89.08
$3.63
$2.86
38.47
61.13
68.81
10.72
59.25
62.15
56,44
44.72
TENT=
35.19
38.12
37.43
34.82
34,95
50.33
54.24
53.88
46.
61,50
69.07
29.77
50.01
62.61
57.09
48.35
HLR (ours)
43.37
44.04
45.68
41.74
41.09
$1.26
56.43
$7.03
50.81
63.115
68.29
50.9%
61.15
63:08
58.15
$3.0
SLR (oum) 41.52
42,90
44.07
41.80 41.89
40.76
54.76
56.39
$7.35
$1.01
63.53
68.73
50.65
61.49
63.46
58.32
53.12
Groundrruth 55.68
38.10
01:27
55.84
55.05
05.83
67.22
67.56
62. 62.60
72.49
76.97
65
70.86
72,51
08.55
05.04
Note that all the hyperparameters are tuned solely on the validation corruptions of ImageNet-C that are disjoint from the test corruptions. As discussed in Section 3.2.2] we freeze all trainable parameters in the top layers of the networks to prohibit "logit explosion" Normalization statistics still updated in these layers. Sec. A.3 provides more details regarding frozen layers in different networks.
Furthermore, we prepend trainable input transformation module d (ef.. Sec. 3.1 the network to partially counteract the data-shift. Note that the parameters of this module discussed in Sec. 3e. are trainable and subject to optimization. This module initialized to operate as identity function prior to adaptation on a target distribution by choosing T = 1, 7 1, and / O. We adapt the parameters of this module along with the thannel-wise affine transformations and normalization statistics in an end-to-end fashion, solely using our proposed loss function along with the optimization details mentioned above. The architecture of this module discussed in Sec.
Since Lav is independent of Leomt, we also propose to combine Lav with TENT, i.e. I. Ladv + Leent- We denote this as TENT+ and also set 0.9 here. Note that TENT optimizes all channel-wise affine parameters in the network (since entropy is saturating and does not cause logit explosion). For a fair comparison to our method, we also freeze the top layers of the networks in TENT+. We show that adding Law and freezing top layers significantly improves the networks performance over TENT. Note that SHOT (Liang et al., 2020) is the combination of TENT, batch-level diversity regularizer, and their pseudo labeling strategy. TENT+ can be seen variant of SHOT but without the pseudo labeling. Please refer to Sec. A.5 for the test-time adaptation of pretrained models with SHOT.
Note that each corruption and severity in ImageNet-C is treated as different target distribution and we reset model parameters to their pretrained values before every adaptation. We run our experiments for three times with random seeds (2020, 2021, 2022) in PyTorch and report the average accuracies.
5 RESULTS
Evaluation on ImageNet-C We adapt different models on the ImageNet-C benchmark using TENT, TENT+, and both hard likelihood ratio (HLR) and soft likelihood ratio (SLR) losses in an online adaptation setting. Figure 2 (top row) depicts the mean corruption accuracy (mCA%) of each model computed across all the corruptions and severity levels. It can be observed that TENT+ improves over TENT, showcasing the importance of diversity regularizer Ldiv- Importantly, our methods HLR and SLR outperform TENT and TENT+ across DenseNer1 21, MobileNetV2, ResNet50, Re5NeX150 and perform comparable with TENT+ on robust ResNet50-DeepAuggmett -Augmix model. This shows that the mCA% of robust DeepAugment+ Augmix model can be further increased from 58% (before adaptation) to 67.5% using test-time adaptation techniques. Here, the average of mCA obtained from three different random seeds are depicted along with the error bars. These smaller error bars represent that the test-time adaptation results are not sensitive to the choice of random seed.
We also illustrate the performance of ResNet50 on the highest severity level across all 15 test corruptions of ImageNet-Chi in Table[]] Here, online adaptation results along with the offline adaptation on epoch 1 and 5 are reported. It can be seen that online adaptation and single epoch of test-time
7

---

Under review as a conference paper at ICLR 2022
Denceflet122
NobielletV2
Recheti50
ResNec50
Reddled DwweStDDDDeeeeeee www.pegggg...............
67.2 22.55
57.9 58.5
61.1 61.2
63.0
56.0
38.1
80.2
S8.0
47.6 49.2 51.5
30
Imagetic
42.1
3944
57.3 57.5
magohett auuuaayyyyyyy 184
50.3 50.6
53.6 53.4
43.9
44,0
40
37.2
34.3
37.4
36.2
Imagotes Ooan kocusscy
77.6
77.6
76.6
75.8
76.1
75.9
75.9
75.8
74
74.4
74.5
74,6
73.2
74.6
74.7
74.0
72
71.2
70.0
Figure fest-time adaptation results on (top row) ImageNet-C, averaged across all 15 corruptions and severities, (middle row) ImageNet-R, (bottom row) clean ImageNet. NA refers to "No Adaptation".
cornuption trast spiit_t:
comupttion glass blur split_byi
corruption: contrast spit_by:
cornuption gaussan poise split_by:
40
50
10
No Adsptation
TENT
spii: samples
by: samples
spilt by samples
split by: samples
TENT+
HLR
40
50
10
0.2
0.4
0.8
0.88
3.5
0.2
0.44
03.5
0.8
2.0
0.2
0.6 Split Fraction
0.4
0.5
1.0
0.2
0..4
0.5
0.21
1.0
Split Fractionn
Split Fraction
Split Fraction
Figure 3: Test-time adaptation of ResNet50 using (top row) a subset of classes, and (bottom row) subset of samples per class on 4 different corruptions at severity 5. Accuracy is computed based on the evaluation of adapted model on the entire target data. Note that error bars are smaller visualize.
adaptation improves the performance significantly and makes minor improvements until epoch 5. TENT adaptation for more than one epoch result in reduced performance and TENT with Laiv (TENT+) prevents this behavior. Both HLR and SLR clearly and consistently outperform TENT / TENT+ on the ResNet50 and also note that SLR outweighs HLR. We also compare our results with the hard pseudo-labels (PL) objective and also with an oracle setting where the groundtruth labels of the target data are used for adapting the model a supervised manner (GT). Note that this oracle setting is not of practical importance but illustrates the empirical upper bound on fully test-time adaptation performance under the chosen modulation parametrization.
Netg We online adapt different models ImageNet-R R and depict the results in Figure (middle row). Results show that HLR and SLR clearly outperform TENT and TENT+ and significantly improve performance of all the models, including the model pretrained with D Augmix.
Evaluation with data subsets Above we evaluate the model on the same data that is also used for the test-time adaptation. Here, test model generalization by adapting subset of target data
8

---

Under review as a conference paper at ICLR 2022
Table 2: SSIM and SLR-adapted ResNet50 accuracy without and with input transformation (IT).
Comuptoon
Gauss
Shot
Impulse
Defocus
Mution
Zuom
Snow
Frost
Fog
Bright
Contrast
Elsttic
Pixul
JPBG
SSIM
0.123
0.147
0.135
0.623
0.648
0.622
0.676
0.517
0.575
0.619
0.655
0.545
0.625
0.786
0.800
SSIMHIT
0.173
0.188
0.347
0.605
0.658
0.603
0.670
0.580
0.628
0.626
0.676
0.765
0.616
0.776
0.795
SLR
41.52
44.07
54.76
56.59
57.35
51.01
63.53
50.65
58.32
SLR-IT
43.09
44.39
64.05
41.98
40.99
55.73
56.75
58.56
$1.68
63.64
68.85
55.01
61.32
63.59
58.24
and evaluate the performance on the whole dataset (in offline setting), which also includes unseen data that is not used for adaptation. We conduct two case studies: (i) adapt on the data from a subset of ImageNet classes and evaluate the performance on the data from all the classes. (ii) Adapt only on a subset of data from each class and test on all seen and unseen samples from the whole dataset.
Figure illustrates generalization of a ResNet50 adapted on different proportions of the data across different corruptions, both in terms of classes and samples. We observe that adapting model on a small subset of samples and classes is sufficient to achieve reasonable accuracy on the whole target data. This suggests that the adaptation actually learns to compensate the data shift rather than overfitting to the adapted samples or classes. The performance of TENT decreases as the number of classes/samples increases, because Lent can converge to trivial collapsed solutions and more data corresponds to more updates steps during adaptation. Adding Lai such as in TENT+ stabilizes the adaptation process and reduces this issues. Reported are the average of random seeds with error bars.
Input transformation We investigate whether the input transformation (IT) module, trained end-to- end with ResNet50 and SLR loss on data of the respective distortion without seeing any source (undistorted) data, can partially undo certain domain shifts of ImageNet-C and also increase accuracy on corrupted data. We measure domain shift via the structural similarity index measure (SSIM) (Wang et al., 2004) between the clean image (unseen by the model) and its distorted version/the output of IT on the distorted version. Following offline adaptation setting, Table 2 shows that IT increases the SSIM considerably on certain distortions such as Impulse, Contrast, Snow, and Frost. IT increases SSIM also for other types of noise distortions, while it slightly reduces SSIM for the blur distortions, Elastic, Pixelate, and JPEG. When combined with SLR, IT considerably increases aceuracy on distortions for which also SSIM increased significantly (for instance +20 percent points on Impulse, +4 percent points on Contrast) and never reduces accuracy by more than 0.1 percent points. More results on online and offline adaptation with TENT/ / TENT+ can be found in Table A3
Clean images As a sanity check, we investigate the effect of test-time adaptation when target data comes from the same distribution as training data. For this, we online adapt pretrained models on clean validation data of ImtgeNet. The results in Figure 2 (bottom row) depict that the performance of SLR/HLR adapted models drops by 0.8 to 1.8 percent points compared to the pretrained model. We attribute this drop self-supervision being less reliable than the original full supervision on in- distribution training data. The drop is smaller for TENT and TENT+, presumably because predictions on in-distribution target data are typically highly confident such that there little gradient and thus little change to the pretrained networks by TENT. In summary, while self-supervision by confidence maximization is powerful method for adaptation domain shift, the observed drop when adapting to data from the source domain indicates that there is "no free lunch" in test-time adaptation.
6 CONCLUSION
We propose a method to improve corruption robustness and domain adaptation of models in a fully test-time adaptation setting. Unlike entropy minimization, our proposed loss functions provide non-vanishing gradients for high confident predictions and thus attribute to improved adaptation in a self-supervised manner. We also show that additional diversity regularization on the model predictions is crucial to prevent trivial solutions and stabilize the adaptation process. Lastly, introduce a trainable input transformation module that partially refines the corrupted samples to support the adaptation. We show that our method improves corruption robustness on ImageNet-C and domain adaptation to ImageNet-R on different ImageNet models. We also show that adaptation on a small fraction of data and classes is sufficient to generalize to unseen target data and classes.
9

---

Under review as a conference paper at ICLR 2022
7 ETHICS STATEMENT
We abide by the general ethical principles listed by ICLR code of ethics. Our work does not include the study of human subjects, dataset releases, do not raise pontential conflicts of interest, or discrininatioo/biia/aaaann
concerns, privacy and security issues. Our non-saturating loss increases aceuracy but might result over confident predictions, which can cause harm in safety- critical downstream applications when not properly calibrated. At the same time, self-supervised confidence maximization might amplify bias in pretrained models. We hope that the diversity regularizer in the loss partially compensates this issue.
REPRODUCIBILITY STATEMENT
We provide complete details of our experimental setup for reproducibility. Sec. provides details of the network architectures, optimizer, learning rate, batch size, choice of hyperparameters of our method and the random seeds used for generating the results. Sec. | provides more details regarding frozen layers in different networks. Sec. A.2 shows the structure of input transformation module used this work. We will also provide a link to an anonymous downloadable source code as comment directed to the reviewers and area chairs in the discussion forum.
REFERENCES
Philipp Benz, Chaoning Zhang, Adil Karjauv, and In So Kweon. Revisiting batch normalization for improving corruption robustness. In Proceedings of the IEEE/CVE Winter Conference on Applications of Computer Vision (WACV), pp. 494-503, January 2021.
Minghao Chen, Hongyang Xue, and Deng Cai. Domain adaptation for semantic segmentation with maximum squares loss. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2090-2099 2019.
Nic Ford, Justin Gilmer, Nicolas Carlini, and Dogus Cubuk. Adversarial examples are natural consequence of test error in noise. arXiv preprint arXiv:1901.10511 2019.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pp. 18001189, PMLR, 2015.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand. and Victor Lempitsky. Domain-adversarial training of neural networks. The journal of machine learning research, 17(1):2096-200 2030, 2016.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, 2019.
Dan Hendrycks and Thomas G. Dietterich. Benchmarking neural network robustness common corruptions and perturbations. International Conference on Learning Representations (ICLR), 2019.
Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshmi- narayanan. Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint arXiv: 2019.
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song. Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: critical analysis of -uttr--distriuution generalization, 2020.
Judy Hoffman, Eric Tzeng Taesung Park, Jun- Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell. Cycada: Cycle-consiseee adversarial domain adaptation. In International conference on machine learning, pp. 1989-1998. PMLR, 2018
10

---

Under review as a conference paper at ICLR 2022
Christoph Kamann, Burkhard Güssefeld, Robin Hutmacher, Jan Hendrik Metzen, and Carsten Rother. Increasing the robustness of semantic segmentation models with painting- cainting-by-numan by-numeers. arXiv preprint arXiv:2010.05495, 2020.
Jogendra Nath Kundu, Naveen Venkat, Rahul M V, and R. Venkatesh Babu. Universal source-free domain adaptation. In Proceedingg of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Vinod K. Kurmi, Venkatesh K. Subramanian, and Vinay P. Namboodiri. Domain impression: A source data free domain adaptation method. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), January 2021.
Bo Li, Yezhen Wang, Tong Che, Shanghang Zhang, Sicheng Zhao, Pengfei Xu, Wei Zhou, Yoshua Bengio, and Kurt Keutzer. Rethinking distributional matching based domain adaptation. arXiv preprint arXivv2006.113: 13352, 2020a.
Rui Li, Qianfen Jiao, Wenming Cao, Hau-San Wong, and Si Wu. Model adaptation: Unsupervised domain adaptation without source data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), June 2020b.
Yanghao Li, Naiyan Wang, Jianping Shi, Jiaying Liu, and Xiaodi Hou. Revisiting batch normalization for practical domain adaptation. arXiv preprint arXiv:1603.nm 04779, 2016.
Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need to access the source data? Source hypothesis transfer for unsupervised domain adaptation. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp.
PMLR, 13-18 Jul 2020.
Raphael Gontijo Lopes, Dong Yin, Ben Poole, Justin Gilmer, and Ekin D Cubuk. Improving robustness without sacrificing accuracy with patch gaussian augmentation. arXiv preprint crXiv:1906.026// 2019.
Fabio Maria Carlucci, Lorenzo Porzi, Barbara Caputo, Elisa Ricci, and Samuel Rota Bulo. Autodial: Automatic domain alignment layers. In Proceedings of the IEEE International Conference on Computer Vision, pp. 5067-5075, 2017.
Eric Mintun, Alexander Kirillov, and Saining Xie. On interaction between augmentations and corruptions in natural corruption robustness. arXiv preprint arXiv:2102.11273,, 202
Eric Mintun, Alexander Kirillov, and Saining Xie. On interaction between augmentations and corruptions in natural corruption robustness, 2021b. URL httpps///penreevee...rttt
2orum?1d==bEupptt.PP
Zachary Nado, Shreyas Padhy, D. Sculley, Alexander D' Amour, Balaji Lakshminarayanan, and Jasper Snoek. Evaluating prediction-time batch normalization for robustness under covariate shift, 2021.
Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. Visda: The visual domain adaptation challenge. arXiv preprint arXiv:1710.06924. 2017.
Joaquin Quiñonero-Candell,, Masashi Sugiyama, Anton Schwaighofer, and N Lawrence. Covariate shift and local learning by distribution matching. 2008
Joaquin Quionero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil Lawrence. Dataset Shift in Machine Learning. MIT Press, 2009.
Evgenia Rusak, Lukas Schott, Roland Zimmermann, Julian Bitterwolf, Oliver Bringmann, Matthias Bethge, and Wieland Brendel. Increasing the robustness of dnns against image corruptions by playing the game of noise. arXiv preprint arXiv:2001.06057.1 2020a.
Evgenia Rusak, Lukas Schott, Roland S. Zimmermann, Julian Bitterwolf, Oliver Bringmann, Matthias Bethge, and Wieland Brendel. A simple way to make neural networks robust against diverse image corruttions. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan- Michael Frahm (eds.), Computer Vision ECCV, 2020b.
11

---

Under review as a conference paper at ICLR 2022
Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann. Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. Advances in Neural Information Processing Systems, 33, 2020.
Baochen Sun, Jiashi Feng, and Kate Saenko. Correlation alignment for unsupervised domain adaptation. In Domain Adaptation in Computer Vision Applications, pp. 153-171. Springer, 2017.
Yu Sun, Eric Tzeng, Trevor Darrell, and Alexei A Efros. Unsupervised domain adaptation through self-supervision. arXiv preprint arXiv: 1825, 2019.
Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexci Efros, and Moritz Hardt. Test-time training with supervipnn for generalization under distribution shifts. In International Conference Machine Learning, pp.
PMLR, 2020.
Hossein Talebi and Peyman Milanfar. Learning to resize images for computer vision tasks. arXiv preprint arXiv:2103.09900 2021.
Torch-Contributors. Torchvision models, 2020. URL httppss/pyyorrdh.or///iiss
sttable/mode 1s.hell
Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.
7167-7176, 2017.
Tuan-Hung Vu, Himalaya Jain, Maxime Bucher, Matthieu Cord, and Patrick Pérez. Advent: Adver- sarial entropy minimization for domain adaptation in semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2517-2526, 2019.
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006.10721 2020.
Zhou Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: From error visibility to structural similarity. Trans. Img. Proc., 13(4))600-612, April 2004. ISSN 1057-7149.
Xiaofu Wu, Quan Zhou, Zhen Yang, Chumming Zhao, Longin Jan Latecki. et al. Entropy minimization vs. diversity maximization for domain adaptation. arXiv preprint arXhv:2002.0169) 2020.
Yanchao Yang and Stefano Soatto. Fda: Fourier domain adaptation for semantic segmentation. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pp.
4085 4095, 2020.
Hengshuai Yao, Dong-lai Zhu, Bei Jiang, and Peng Yu. Negative log likelihood ratio loss for deep neural network classification. In Kohei Arai, Rahul Bhatia, and Supriya Kapoor (eds.), Proceedings of the Future Technologies Conference (FTC), pp. 276-282, Cham, 2020. Springer International Publishing. ISBN 978-3-030-32520-6
Hao-We Wei Yeh, Baoyao Yang, Pong C. Yuen, and Tatsuya Harada. Sofa: Cource-data-fis feature alignment for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), January 2021.
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6023-6032 2019.
12

---

Under review as a conference paper at ICLR 2022
A APPENDIX
A.1 ILLUSTRATIVE EXAMPLE OF LOG LIKELIHOOD RATIO ADAPTATION OBJECTIVE
A simple ID example is devised to illustrate the benefits of proposed log likelihood ratio as test time adaptation objective. Consider data points (unlabeled) that are sampled from the following bimodal distribution: 0.5 (- + 0.5 ((+1,3) that is: half of the samples come from a normal distribution with mean -1 and the other half from normal distribution with mean +1 (and both having standard deviation 3). We can interpret these two components of the mixture distributions as corresponding to data of two different classes, but class labels are of course unavailable during unsupervised rest-time adaptation.
We assume a simple logistic model of the form po 1|2) 1+e-(1+0)) where is the value of the data sample and is a scalar offset that determines the decision boundary. By construction, we know that the minimum density of the mixture distribution on [-1, 1] is at O. Since confidence maximization aims as moving the decision boundary to regions in input space with minimum data density (in this case to 0), we can compare different self-supervised confidence maximization losses in the finite data regime as follows: for every finite data sample with data points {o;} for i = J,...,,. N and loss function L, we solve 0* (L) arg min 1,1] L(e, {x,)), where the loss (such entropy SLR)
is averaged over all data points. The absolute value 10 (I.) gives then an estimate of the error of the decision boundary parameter (0" (L) for the given data set and loss function.
Table Al provides this error for different loss functions and different number of data samples. It can be seen that SLR and HLR clearly outperform Entropy loss (TENT) for all data regimes. The difference between SLR and HLR is generally very small. While SLR seems to be consistently slightly better than HLR, this difference is not statistically significant. We attribute the superiority of SLR/HLR compared to entropy to the fact that all data points have non-saturating loss, regardless of their distance to the decision boundary. Thus, all data contributes to localizing the decision boundary, while for saturating losses such as the entropy, effectively only "nearby" points determine the decision boundary. This example illustrates that our proposed non-saturating losses are beneficial over entropy loss for self-supervised confidence maximization.
Table A1: Illustrates the error of the decision boundary parameter for different loss functions and different number of samples averaged over 100 runs (shown are mean and standard error of mean).
#samples
100
200
500
1000
2000
10000
20000
Entropy
0.487:10.031
0.36410.029
0.230=00018
0.152:10.013
0.117:10.099
0.052=0.004
0.033100003
HLR
0.357
0.234
0.145=0.012
0.094=0.008
0.071=00006
0.032=0.002
0.022=0.002
SLR
0.332:10002
0.214
0.140=0.011
0.088=0.008
0.067=20006
0.032-0.002
0.021=0.002
A.2 INPUT TRANSFORMATION MODULE
Note that we detine our adaptable model as g f o d. where d a trainable network prepended to a pretrained neural network f (e.g.. pretrained ResNet50). We choose d(x) = 7-[Tz + (1 - (z)] 3, where T E IR, (8,7) E RR with ni.n being the number of input channels, r. being network with identical input and output shape, and denoting elementwise multiplication. Here, / and y implement channel- wise affine transformation and implements a convex combination of unchanged input and the transformed input rw (2r). We set = 1, = 1, and 3 = O, to ensure that d(x:) = x and thus g = I at initialization. In principle, re can be chosen arbitrarily. Here, we choose ru as a simple stack of 3 x 3 convolutions with stride 1 and padding 1, group normalization, and ReLUs without any ppsampiig//oownsamiing layers. Specifically, the structure of g is illustrated in Figure Al
In addition to the results reported in Table 2| we also compare TENT and TENT+ with and without Input Transformation (IT) module on ResNet50 for all corruptions at severity level 5 in both online adaptation setting and offline adaptation with 5 epochs in Table A3 Furthermore, we also present the qualitative results of the image transformations from the input transformation module adapted with SLR (offline setting) in Figure A2
13

---

Under review as a conference paper at ICLR 2022
Table A2: Ablation study on the components of input transformation module on ResNet50 for all corruptions at severity level 5.
Comppton
Gouss
Shot
Inpulse
Defocus
Glass
Motioz
Zoomt
Snow
Froxt
Fog
Brigt
Coutnst
Elastic
Pioel
JPEG
men
44.07
$7.35
51.01
63.53
57
28.81
53.73
45.96
58.82
56.77
63.05
46.43
56.25
41.30
40.90
56.05
63.59
61.50
43.18
46.24
56.21
40.89
56.66
38.00
51.72
6336
68.1
54,26
61.49
Table A3: Test-time adaptation of ResNe150 on ImageNet-C at highest severity level 5 with and without Input Transformation (IT) module. Reported are the mean accuracy(S)) across three random seeds 22000/221/20222. While also improves performance when combined with TENT+, is still clearly outperformed by SLR+IT.
Method
Gauss
Shot
Impulse
Defocus
Glass
Metion
Zoom
Snow
Frost
Bright
Contratt
Elastic
Pixel
JPEG
Culline adaptation (exaluriin en a hatch direcrly after odaptation batch)
TENT
28.50
31.05
30.54
29.09
28.07
42,32
50.39
48.01
42,05
58.40
68.20
27.25
55.68
59.46
53.64
TENT + IT
28.99
51.73
31:15
28.87
27.85
42.43
50.36
48.02
41.95
58.57
68.19
24.35
55.68
59.49
53.57
TENT=
29.09
31.65
30.68
29.33
28.65
42.32
50.32
48.09
42.54
58.39
68.23
31.43
$5.90
59.46
53.68
TENT- 29.48
32.34
31.38
29.06
28.42
50.33
48.11
58.40
68.20
32.11
55.87
59.49
53.64
ST.
35.13
58.74
55.97
SLR (ouas)
34,87
53.62
52.71
58.79
61.27
55.93
Evalustion afler spoch
TENT
30.64
$3.80
34.72
30.13
29.05
49.08
$3.63
$2.86
38.47
61.13
68.81
10.72
59:25
62.15
56.44
TENT a IT
31.92
36,02
38,
30.44
28.68
49.04
53.59
52.99
38.76
61.14
68.34
13.52
99:23
62.15
56.56
TENT-
35.19
38.12
37.43
34.82
34,95
50.33
54,24
53.88
61.50
29.87
00.01
62.61
57.09
TENT-
41
34.62
50.33
54.10
59.95
57.11
SLR (ouns)
40.78
56.59
50.65
61.49
SLR-IT (ours)
43.09
44.39
64.05
41.98
40.99
55.73
56.75
S8.56
51.68
63.64
68.85
55.01
61.32
63.59
58.24
A.2.1 CONTRIBUTION OF EACH COMPONENT INPUT TRANSFORMATION MODULE
Table A2 shows the results of ablation study on the components of input transformation module on ResNet50 for all corruptions at severity level 5 adapted with SLR for 5 epochs. The ablation study includes: (I) no input transformation module d(x) = x, (2) with network d(x) = r (z), (3) including (4) including channel-wise affine transformation y and B. We can observe that the inputs transformed with network ry drops the performance without the convex combination with т. The additional channel wise affine transformations didn't 't bring further consistent improvements and can be ignored from the transformation module. Exploring other architectural choices and training (or pretraining) strategy for the input transformation module would be interesting avenue for future work.
A.3 FROZEN LAYERS IN DIFFERENT NETWORKS
As discussed in Section 3.2.2 we freeze all trainable parameters in the top layers the networks to prohibit "logit explosion" That implies, we do not optimize the channel-wise affine transformations of the top layers but normalization statistics are still estimated. Similar to the hyperparameters of test time adaptation settings, the choice of these layers are made using ImageNet-C validation data. We mention the frozen layers of each architecuture below. Note that the naming convention of these layers are based on the model definition in torchvision:
DenseNet121 feeatures.deneeblocke features.norm5.
MobileNetV2 features. features. features.
ResNeX150. ResNet50 and ResNet50 (DeepAuugment+Auugiis layer4.
A.3.1 RESULTS WITHOUT FREEZING THE TOP LAYERS
We mentioned that the proposed losses could alternatively encourage the network to scale the logits grow larger and larger and still reduce the loss. However, we did not find any considerable differences empirically in the explored settings when adapting the model with or without freezing the top layer. We found that adapting the model with and without freezing the top layers have comparable performance in both online and offline adaptation settings as shown in Table/A4/reppectvel would still recommend freezing the top-most layers the default choice to be the safe side. These results indicate that the early layers capture the distribution shift sufficiently to improve the model adaptation.
However,
14

---

Under review as a conference paper at ICLR 2022
Table A4: Comparing the online and offline adaptation results with and without freezing the afline parameters of top normalization layers of ResNet50 at severity 5. Here, "Freeze" and "NoFreeze" refer to the setting with and without freezing the top affine layers respectively.
Comppton
Gauss
Inquise
Defocus
Mation
Foot
Bright
Connes
Elastic
Pixel
JPBG
Ouline cvaluation
TENT- NeFrase
29,05
28.95
17.37
48.12
42.21
68.29
28.17
55.57
TENT+
29.17
28.60
48.38
42.51
68,30
31.25
Nofreeae
38.73
36.50
33.99
33.88
46.55
51.44
45.5
67.37
43.19
54.95
47.53
HIL
51.42
45.47
42.75
47.40)
NaFreasee
35.61
35.83
35.81
46.
52.62
46.85
67.71
44.93
58.45
55.65
48.8
SLR Freese
35.11
36,83
35.13
53.45
52.68
44.78
55,97
ovalustion
TENT- Nefrerse
$2.03
55.35
15.28
11:92
31.27
53.01
40
79.79
10. 58
90.30 25
62.20
$6.51
45.07
TENT+ Froeze
$5.19
34.82
57,09
NelFoeee
41.00
43.50
40.21
41.50
53.82
56.21
56.71
90.85
51.54
62.58
57,00
52.89
[ILR Freeze
41.37
44.04
43.08
41.74
54.26
56.43
57.00
50,81
08.29
50.98
61.15
58.13
53.0
SIR NaFrranee
41:45
43.05
42.56
41
54.25
56.72
90.92
50.99
53.02
SUR Treese
41.52
44.07
41.89
54.76
57.35
51.01
63.53
50.85
58.32
A.4 EFFECT OF K
Note that the running estimate of Ldiv prevents model collapsed to trivial solutions i.e., model predicts only a single or a set of classes as outputs regardless of the input samples. Lav encourages model to match it's empirical distribution of predictions to class distribution of target data (uniform distribution in our experiments). Such diversity regularization is crucial as there is no direct supervision attributing to different classes and thus aids to avoid collapsed trivial solutions. In Figure A33 we investigate different values of к on validation corruptions of ImageNet-C to study its effectiveness on our approach. It can be observed that both the HLR and SLR without Lai- leads to collapsed solutions (e.g., accuracy drops to 0%) on some of the corruptions and the performance gains are not consistent across all the corruptions. On the other hand, Lav with 0.9 remain consistent and improve the performance across all the corruptions.
A.5 TEST-TIME ADPTATION OF PRETRAINED MODELS WITH SHOT
Following SHOT (Lang et al., 2020), we use their pseudo labeling strategy on the ImageNet pretrained ResNet50 in combination with TENT+, HLR and SLR. Note that TENT+ and pseudo labeling strategy jointly forms the method SHOT. The pseudo labeling strategy starts after the 1st epoch and thereafter computed at every epoch. The weight for the loss computed the pseudo labels is set to 0.3, similar to (Liang et al.] 2020). Different values for this weight is explored and found 0.3 perform best. Table A6 compares the results of the methods with and without pseudo labeling strategy. It can be observed that the results with pseudo labeling strategy perform worse than without taking this strategy into account.
We further modified the pretrained ResNet50 by following the network modifications suggested in (Liang et al.) 2020), that includes adding bottleneck layer with BatchNorm and applying weight norm on the linear classifier along with smooth label training to facilitate the pseudo labeling strategy. Table 7 shows that the pseudo labeling strategy on such network improve the results of TENT+ from epoch 1 to epoch 5. However, there are no improvements noticed in SLR. Moreover, Table A8 shows that NO pseudo labeling strategy on the same network performs better than applying the pseudo labeling strategy. Finally, the no pseduo labeling results from Table A6 A8 additional modifications to ResNet50 do not improve the performance when compared to the standard ResNet50.
A8 shows that
A.6 DOMATN ADAPTATION ON VISDA- C AND DIGIT CLASSIFICATION
We extended our experiments to VisDA-C. We followed similar network architecture from SHOT (Liang et al. 2020) and evaluated TENT+, our SLR loss function with diversity regu- larizer. Similar to ImageNet-C, we adapted only the channel wise affine parameters of batchnorm layers for 5 epochs with Adam optimizer with cosine decay scheduler of the learning rate with initial value 2e- - 5. Here, the batchsize is set to 64, the weight of Loouf in our loss function to 5 = 0.25 and K = in the rumming estimate (t()) of Laiv. since the number of classes in this dataset (12 classes) is smaller than the batchsize. Setting 0 enables the batch wise diversity regularizer. Table A9
VisDA-C:
15

---

Under review as a conference paper at ICLR 2022
Table AS: Test-time adaptation of ResNet50 on ImageNet-C highest severity level 5, Table [ with error bars.
Same as
name
Epuch!
Spoch
comupption
No athpiaatio
PL
TENT
TENT+
BLR
SLR
TENT
TENT+
HLR
SLR
2.44
52.44=0.11
33. 09
08.20
39.51=0.25
30.64=0.51
35.19+0.17
41.37=0.09
41.52+0.08
26.38=0.15
41.11
18.1240.10
42.90=20.08
Impulse
1.96
1.96
34.77.0.09
35.67
20.28.10.20
48.08.10.4
34.72.1.00 101
37.4310.09
43.68.00.06
44.07.0.006
Defocus
17.92
17.92
52.40_0.110
33.4310.14
38.2510.32
39.3510..13
30.1310611
34.8210.25
41.74
41.9910.07
Glass
9.82
988
51 1.115
33 03.25:0001
18.18:00.08
39.02:00009
24 05 0.33 01
34.9510.13
41.09:0.17
40.78:10.08
Mation
47.25-0.11
51.63+0.08
09.08-0008
50.33+00.07
04.36-0002
54.76+0.04
56.59.10.05
Snow
16.89
51.61:0..55
52.06.10.09
55,45110.11
55.92:00.06
52.86.10.13
53.88.10.07
57.03:00112
57.3610.03
Froet
23.31
23.31
43.26+00.30
44:55=0
48, 46.96++0.07
49.64-0.114
38 0.
46. 46.28+00.27
50.81=0000
51.01-0.022
Fag
24.41
24.43
6042++0.06
00.60=00.05
62.19+0.03
62,62-0.044
61. 03-00
61. 61.50+00.05
63.05=0004
63.53+0.08
Bright
58.93
58.93
68.85 8500020
68.93=0.3
68.17:0.01
68.4==0.55
08.81=0.06
69.07
68.29=0.09
68.72=0.10
Contrast
33.43
49.4740.20
50.27:0.05
10:72=0.32
29.8741..36
Elasric
16.95
60.34+0.11
60.00-008
59
60.0++0:2
61.49+0.77
Pixel
20.61
20.61
61.62-0.06
61.75=007
62.51+0.10
6301=0008
62. 15=0004
62.61+0.08
63.08-0.066
63.46+00008
JPEG
31.65
31.65
56.00=00.99
56.21=0005
57.4220.13
57.50=0.044
56.44:0.07
57.09
58. 13:0090009
58.32
Table A6: Test-time adaptation of ResNet50 on ImageNet-C at highest severity level 5 with and without the pseudo labeling strategy (Liang et al.. 2020).
name
No pseudo labeling: Epoch
Pseudo labeling: Epoch 5
corruption
No adaptation
TENT+
HLR
SLR
TENT+
HLR
SLR
Gauss
2.44
33.97:0.17
41.37=0.09
41.52:0.08
34.08
34.88=0.35
35.5810.006
Shot
2.99
37.95 10
4.00
42.90:10.08
36.74.10.26
37.61=0.49
37.98:10.19
Impulsc
1.96
36.93++0.09
43.68=0.06
44.07=40.06
36.69=1004
37.24=0.22
37.77=0.005
Defocus
17.92
32.69:1..25
41.74-0.12
41.6910.07
33.99110.28
34.76-0.11
35.1110.110
Glass
9.82
33.36110.1
41.09=0.17
40.7810.08
34.06:10.12
34.51=0.30
34.81+0.27
Motion
14.78
51,42.10.07
54.26.1.02
54.76.10.04
50.91110.09
48.961..39
49.46.10.20
Zoom
22.50
54.33+0.066
56.43=0.07
56.59+0.05
54.10=0.110
52.49=0.02
52.50++0.23
Snow
16.89
54.55110.07
57.03=0.12
57.3510.03
54.06.10.08
$2.49=0.19
52.9510.07
Frost
23.31
45.80++0.27
50.81+00088
51.01+0.02
44.44+10.07
45.47+0.26
06.06+00.00
Fog
24.43
62.0910.005
63.0510.04
63.53.10.08
61.91
59.66
59.9810.11
Bright
58.93
69.03++0.06
68.29-0.09
68.72+0.10
68.98+0.02
65.59-0.06
66.00+00003
Contrast
5.43
24.08:11.36
50.98-2.54
50.6510.55
29.37
44.58-0.38
45.6410.47
Elastic
16.95
60.36:10.02
61.15:0004
61.4910.07
60.23:10.05
57.48-0.14
57.87110.04
Pixel
20.61
63.1010.008
63.0810.06
63.46.10.88
62.9811.04
59.72.10022
60.0510.11
JPFG
31.65
57.21+00.2
58.13+0.09
58.32+0.05
57.09+0.04
54.72+0.09
54.88+0.07
average results from three different random seeds and also shows that SLR outperforms TENT+ this dataset.
Domain adaptation from SVHN to MNIST MMNISTM / USPS: ResNet26 is trained on SVHN
dataset for 50 epochs with batch size 128, SGD optimizer with momentum 0.9 and initial learning rate 0.01, which drops to 0.001 and 0.0001 at 25th and 40th epoch respectively. ResNet26 obtains 96.49% test accuracy on SVHN. Domain adaptation of SVHN trained ResNet26 to MNIST/MNIST-M/UUPS
Table A7: Test-time adaptation of modified ResNet50 (following (Liang et al. 2020 on ImageNet-C highest severity level with pseudo labeling strategy at epoch and epoch
name
Pseudo labeling: Epoch
Pseudo labeling: Epoch 5
corruption
No adaptation
TENT+
IIR
SLR
TENT+
ILR
SLR
Gauss
2.95
31.03-10.18
34.65=0.28
37.21+1.233
35.26:10.16
35.93-0.23
37.61110.30
Shot
3.65
33.55=0.007
38.09=0.30
40.30+0.09
37.39=0.05
38.95=0.16
40.42+0.06
Impulse
2.54
32.70.10.07
36.95_0.05
39.73.0.07
38.16110.08
38.1310.04
10.12.1..11
Defocus
19.36
31.66+00.15
35.08-0.05
37.18+0.15
35.95+0.17
36.72-0.13
37.96+0.25
Glass
9.72
31.06=10.06
35.46=0.12
37.62+0.10
35.98-0.04
36.84=0.11
37.9020.02
Motion
15.66
46.96.1..12
49.9510.12
51.87.10.14
52.2410.02
51.90...112
52.76.10.09
Zoom
22.20
52.45.10.02
54.15-0.22
54.8410.118
54.80-10.07
$4.84-0.09
54.9511.114
Snow
17.56
51.79++0.05
53.98-0.06
55.44+0.04
55.15+0.02
55.27+0.20
05.75+0.02
Frost
24.11
45.59-10.06
47.87-0.03
48.9640.11
48.10:10.20
48.52-0.11
49.13-0.20
Fog
25.59
60.33110.3
61.5510.10
62.2110.116
62.3910.:3
62.3810.12
62.38.0.111
Bright
58.30
68.8410.04
68.44_0.04
68.6010.10
69.13.10.04
68.50-0.02
68.47-10.99
Contrast
6.49
42.34+0.119
47.98-0.11
50.32+0.28
42.11-10.15
49.22-0.42
50.80+0.19
Elastic
17.72
58.47+0.02
59.70=0.06
60.30+0.09
60.40=0.04
60.27-0.22
60.45+0.21
Pixel
21.29
61.3911006
62.10
62.7110.100
63.04.10.02
62.71-0007
62.81110007
JPFG
32.13
55.22++0.33
56.49-0.07
$7.04+0.07
57.21
57.25+0.07
$7.37+0.05
16

---

Under review as a conference paper at ICLR 2022
Table A8: Test-time adaptation of modified ResNet50 (following (Liang et 2020 on ImageNet-C at highest severity level 5 with and without pseudo labeling strategy.
No Pseudo labeling: Epoch 5
Pseudo labeling: Epoch 5
corruption
adaptation
TENT+
HLR
SI.R
TENT+
HLR
SI.R
Gauss
2.95
34.96:10.08
38.58=0.12
39.7240.11
35.26=+0.16
35.93=0.23
37.61=0.330
Shot
3.65
37.22:1.17
41.59-0.09
42.45:1.005
37.39:10005
38.95-0.16
40.42.10.06
Impulse
2.54
37.82+00.04
40.88=0.07
42.39+00.3
38.16+0.088
38.13-0.04
10.12+0.111
Defocus
19.36
34.46=00112
39.22=0.15
39.78+0.09
35.95=0.17
36.72=0.13
37.96=0.25
Glass
9.72
35.12+00.05
38.83=0.13
39.37+0.07
35.98+0.04
36.84=0.11
37.90=0.02
Motion
15.66
51.91
53.23=0.05
54.00
52.2410.02
51.90=0.12
52.7610.099
Zoom
22.20
54.57:+0.05
55.76=0.04
55.79+0.02
54.80+0.077
$4.84=0.09
54.95+0.14
Snow
17.56
55.02+0.005
56.35=0.12
56.80=0.04
55.15=0.022
55.27=0.20
55.75=0.022
Frost
24.11
48.18.10.09
49.86-1022
50,43-10008
48.10.10.20
48.52-0.11
49.1310.22
Fog
25.59
62.24:1004
62.90=0.06
63.29::0.06
62.39:10.33
62.38-0.12
62.3840.11
Bright
58,30
69.12.10.01
68.72:0006
68.83110.05
69.13:10.04
68.50:0.02
68.47.10.09
Contrast
6.49
33.91=0.92
52.13=0.16
53.04=0.14
42.11=0.15
49.22=0.42
50.80=0.19
Elastic
17.72
60.3710.11
60.89-0.08
61.1210.011
60,40:1..44
60.27-0.22
60.4510.21
Pixel
21.29
62.97=0.02
62.95=0005
63.21
63.04=0.02
62.71=0.07
62.81=0.07
JPEG
32.13
57.10.00.06
57.91
57.9910.11
57.21
57.2510.07
$7.37
Table A9: Performance on VisDA-C dataset
Method
Accuracy(%)
No Adaptation 46.1
TENT+
81.83=0.16
82.32=0.16
conducted with Adam optimizer with constant learning rate 0.001 for 20 epochs on TENT, TENT+ and SLR with three random seeds (2020/2021 1/2022). Table TableAA10 compares our proposed loss SLR with TENT variants on ResNet26 and shows that our approach outperforms them across all the datasets.
A.7 COMPARISON WITH MAX SQUARE LOSS AND CHARBONNIER PENALTY
2019) and charbonnier Similar to Figure we provide the illustration of max square loss (Chen et al.
penalty applied to entropy minimization Yangg & Soatto 2020) Figure A4 Both the max Squares loss and the charbonnier penalty applied to entropy minimization have gradients of the loss w.r.t. the logit that go to for high confidence predictions (confidences greater than 0.95). This not the case for our proposed non-saturating losses. This vanishing gradient cause high confidence predictions to have less contribution on the test-time adaptation than lower confident predictions. We also compare the online test-time adaptation results of these losses with proposed losses in Table A1b
A.8 TESTS IMAGENET-A
We adapted ResNe150 ImageNet-A dataset for 5 epochs with different losses presented in the Table A12 We see that SLR outperforms the other losses. However, the improvements from test-time adaptation this dataset are minimal when compared to the results corrupted datasets. Thus, test-time adaptation is best suited when there is a shift in appearance in the inputs rather involving high-level semantic changes in ImageNet-A.
Table A10: Digit domain adaptation from SVHN to MNIST / MNIST-M USPS. Reported values are mean accuracy(6)) over three random seeds 2020/2021/20222,
MNIST MNIST-M
USPS
No adaptation
42.48
47.43
11.83
TENT
93.5
56.9
84.0
TENT+ 96.9
67.4
85.6
SLR (ours)
98.3
77.4
94.2
17

---

Under review as a conference paper at ICLR 2022
Table All:
Test-time online adaptation of ResNet50 on ImageNet-C at highest severity level 5. Similar to TENT+, Lav is combined with Max Square loss and Charbonnier Penalty and also freeze top layers of the network. We denote these settings as Max Square+ and Charbonnier Penalty+. Here, different 7 E {0.1,033 0.75, 1.0, 1.75, 2.0] values are explored for Charbonnier penalty and found 1j = 0.3 performs better. Reported values are mean aceuracy over three random seeds (2020/2221/2022) of ResNet50 at severity level 5. Results show that SLR outperforms Max Squares loss and the Charbonnier Penalty, even when tuning the Charbonnier penalty's ' hyperparamter carefully.
Comuptoo
Gauss
Shot
Impuise
Defocas
Glass
Mosicn
Zoom
Snow
Frost
Fog
Bright
Contrast
Elustic
Pixel
IPBG
Mat aguane
17.14
18.50
17.68
17:58
17:40
30002
42.81
38.15
16.03
52.25
75
21.23
47.66
53.30 14.44
45.00
Max squance
16.85
15.15
17.85
12.20
17.08
20:50
42.34
$3.67
35.88
51.74
66.58
20.95
47.28
57.7%
44.46
Clurhesnizz Faully (ctamt)
24.82
27.00
26.03
25.34
24.48
38.36
48.47
<5.40
40.76
56.98
67.90
28.92
53.40
$7.96
$1.38
Charbsonier Panally- (ctamil 3)
24.55
26.76
25.45
24.86
24.23
38,00
48.22
44.95
40.55
56.80
67.83
30.22
53.33
$7.77
51.34
TENT
28 A0
$1.06
30.54
79.09
28.07
42.53
50.19
68.01
42.05
58 40
68.20
27.25
55.68
99:46
TENT+ (oarn)
28.65
48.09
58
19:10
6000
SLR (muns)
35.01
46.52
61.74
44.78
61.13
Table A12: ResNet50 adaptation on ImageNet-A (reported is the model accuracy)
adaptation
TENT TENT+
IILR
SLR
ImageNet-A
0.0%
0.04%
0.08%
0.51%
0.55%
A.9 ADDITIONAL STUDIES ON TENT/TENT
Optimizer and learning rate: The default optimization setting proposed for TENT (Wang et al. 2020) is SGD with learning rate (Ir) 0.00025. Table A13 provide additional results for TENT and TENT+ results with both SGD and Adam at different learning rates for both online and offline (5 epochs) adaptation settings on ResNet50 for all corruptions at severity level 5. We can notice that SGD shown to perform better than Adam for TENT and slightly better for TENT+. Among different learning rates with SGD, higher learning rates bring additional improvements for TENT and TENT+ in online adaptation setting but they still outperformed by our SLR results. However, for offline updates, the higher learning rates shown to hurt the model performance with TENT TENT+ and our SLR results are superior over them. Note that our optimizer and learning rate remain same for both online and offline adaptation settings. Here, TENT behaves differently in online and offline adaptation for the same learning rate and our HLR/SLR behaves the same in both the settings.
Results without freezing top layers: We run TENT by frrezing all but the affine parameters of normalization layers and we freeze top affine layers of the network for TENT+ additionally. In Table A14 present the results of TENT+ without freezing the top affine layers of ResNet50 for all corruptions at severity level with different learning rates using SGD. We can observe that the network with freezing top layers has comparable or sometimes better performance than no freeze network. These results indicate that the distribution shift is better captured in early affine layers and handling the distribution shift deeper affine layers with high learning rate limit their functionality to operate on abstract representations and thereby affect the model performance.
18

---

Under review as a conference paper at ICLR 2022
Table A13: Evaluation of TENT and TENT+ with both SGD and Adam different learning rate on ResNet50 for all corruptions at severity level 5.
Method
Gaues
Shot
Impulse
Defoess
Glass
Motion
7oom
Snow
Frost
Fog
Bright
Contratt
Flaetic
Pisel
JPBC
mesn
adogeation TENT SGD
0.0000
24.76
25.97
25.93
25.13
24.37
37.74
67.98
44.73
40.26
36.43
67.52
27.51
52.88
57.34
50.85
40.7
RO00IES
25.70
34:57
30.38
29.03
25.00
42.46
50.42
48.15
42.03
58.48
68.25
26.97
55.66
59.52
53.54
45.55
Ir 0.000
30.01
32:35
32.17
30.11
29.25
44.30
51.37
49.42
42.18
59.18
68.43
24.19
56.66
60.26
54.45
44.29
Ir 0.0006
30.25
32.90
32.84
30,12
29.10
45.52
51.94
90
41.05
59.55
68.52
19.11
57.34
60.62
54.5 54.92
44.
0.0008
29.59
32.48
32.73
29.37
28.38
16.07
$2.05
$0.59
40.45
59,70
68.48
15.03
57.54
60.78
55.10
43.85
0.0000
32.06
52.03
90.72
43.2%
online odupiation TENT
OL00001
28.60
0L0012
25.06
28.47
29.17
25.50
43.54
49.40
37.51
59.39
68.27
14.83
36.91
60.05
54.37
41.83
Ir 0L0004
18:53
21:34
24.40
19.65
34
41.61
49.67
47.5 14.94
31.70
58.77
67.65
9.30
56.46
59.66
$3.69
38
Ir 0.0006
12.41
14.23
13.49
13:28
10.68
34.22
44.63
41.62
21.65
56.05
66.20
6.31
52.08
57.12
50.41
33.13
Ir 0.080S
8.98
10.05
10.82
9.76
8.08
24.09
$7.15
32.72
15.69
49.32
63. 53.58
4.42
44.82
53.36
42.22
27.65
0,000
6.92
7.57
8.34
7.53
6.41
18.46
28:07
25.61
12.55
40.22
57,01
3.35
35,87
45.79
31,55
22.34
oaline adugtation
TENT+
SGGD
Ir 0.0000
24.73
26.91
25.63
24.56
24.31
37.55
47.79
44:49
40.15
56.30
67.57
28.67
52.80
57.19
50.70
40.64
0.0002
29.21
31.54
10.55
20.17
28.60
42.54
50.47
48. 18
42.51
58. 58.50
68. 30
31.25
55.76
59.54
53.62
43.98
0.0004
30.95
33.35
32.65
30.90
30.43
44.67
51.49
69.63
43.54
59.29
68.45
31.21
56.98
60.37
54.60
45.23
0.000
33.91
52.18
50.69
44.20)
0.0008
000000
32.52
34.84
80
24.75
61.45
46.13
0.002
31.03
34.69
33.39
29.41
29.30
48.12
52.47
52.11
42.96
60:42
68.48
15.25
38.57
61.67
55.74
48.91
online adappation
TENT+
Adam
Ir 0.0001
27.95
30.48
29.30
27.36
21.80
41.57
50.64
48.07
42.44
58.65
68.53
29.24
55.98
19.31
53.38
43.41
Ir OLDBRSS
31.07
33.60
33.10
30.19
50.52
45.54
52. 26.36
50.63
44.12
60.00
68.65
28.
58.02
66.77
55.15
45.49
Ir 0.0004
31.31
33.90
33.59
30.00
30.29
60
52.46
51.41
43.65
60.26
68.43
25.66
58.25
61.08
55.44
45.48
0.0000
32.65
46.35
44.5%
0.000
24.61
42.79
0000001
28.53
27.92
20.30
49. 89
37.94
67.15
9.77
57.11
53.86
40:91
online aliptation Ours Adam Ir 0.0006
HLR
46.36
47
SLR
52.68
otfline odaptation TEN T+ SGD
Tr 0,001
27.81
33.79
12.94
25.
26.52
50.95
52,
54, 19
43.29
62.07
68.78
2.66
60.24
63.06
56.
44.07
0.0000
31.80
35.01
21.64
51.61
53.33
$4.47
44.37
68.89
57.07
45.6
0.0002
37.43
offiine aduptation
Adam
0.0006
HLL
41.37
44.04
43.68
41.74
41.09
54,26
55.43
$7.03
50.81
63,05
68.29
50.98
61.15
63.08
58.13
53.0
SLR
41.52
42.00
44.07
41.69
40.78
54.76
$6.59
$7.35
$1.01
63.53
68.72
50.65
61.49
63.46
$M.32
53.12
Table A14: Evaluation of TENT+ with and without freezing the top affine layers at different learning rate on ResNet50 for all corruptions at severity level Here, "Freeze" and "NoFreeze" refer the setting with and without freezing the top affine layers respectively.
Methed
Gauss
Shot
Impulse
Defocus
Glass
Motion
Zoom
Snow
Frost
Fog
Bright
Contrast
Elbaticc
Pixel
JPBG
mean
online adapeation TENT+ SCD 0.0025
NolFreeze 29,05
30.32
50.55
68.20
43.63
Freeze
29.21
31.54
30.35
5047
48.18
58.50
59.54
online adaptation TENT+
SGD
0.0044
NoPFresze Frenze
30.38
32.86
32.17
30.26
29.64
44
51
$1.42
49.45
43.01
59.12
68.48
27.83
56.75
60.32
$4.42
44.70
30.95
33.35
32.66
30,90
30.43
44.67
49
49.63
43.54
59
68. 48
31.21
56.98
60.37
$4,50
45:23
online adaptation TENT
S030
0.00000
0.0006
NoFFrese Presse
30.87
33.72
32.85
30.58
29.76
45.55
52.01
50.32
43.03
59.47
68.60
25.66
57.35
60.71
51.85
45.02
$2.00
34.66
45.96
online adaptation
NoFreese Freaze
33.58
32.98
29.50
52.1
50.69
59.66
60.85
$2.37
36.16
$4.38
32.00
31.91
46.96
52.50
51.28
44.55
00088
68.68
27.23
58.09
61.21
33.06
46.15
online adaptation TENT+
SCD
0.001
NoFreese Freeze
30.4%
33.07
92.58
29.42
28.66
46.54
$2.09
50.77
41.84
59
68.41
18.57
57.72
60.97
55.13
44,39
32.52
35.32
34.84
31.89
32.00
41
52.56
51.61
44.66
60.24
68.66
24.73
58.34
61,45
55.82
46.
19

---

Under review as a conference paper at ICLR 2022
23,224, 224)
channels
5 X
GroupNorm
T ạp
convy channels
GroupNorm
(1-1)
B
dx)
1
(pretrained neural network) f((())
Figure A1: Structure of our adaptable model that comprises of righ
QUANEES IEEA
UNE FOGS
JO0TH MAY ITD rotoe
SUTCCUS BAX PUSDON
DIGREEASS
CONTRuE
(90] COmBROSOO
SSM 67L
SSIN
Figure A2: Qualitative results of image transformation from input transformation module adapted with SLR.
(8))
corruption: saturate
corruption: spatter
corruction :spocklc_noce
cormuption: goussion blu
(a)
corruption: speckle_rod
Aucutayy (86)
Method
Soft Bethood Bat with Soft Liethood Bai witnout
(b)
Figure A3: Effect different on both (a) HLR and (b) SLR
20

---

Under review as a conference paper at ICLR 2022
0.0
-.22
Logit Gradient 0L
80:
0.2
Loss L
-0.4
0.0
-0.6
Loss 1,
Hard Likellhood Ratio
-0.8
Liulitoud
0.2
Entropy
MS
-1.0
0.0
0.2
0.4
0.8
1.0
0.0
0.4
0.6
0.8
Confidence y/
Confidence y1
(a)
0.0
-.2
Logit Gradient OL
dol
-0.4
Loss
Loss L
-0.6
Soft Likelhood Ratio
Entropy
Charbone (EEnrropy eta=0
-0.8
-0.2
Charbone e (Entropy, eta=00 90)
Charbone er (Entropy, eta=00 99)
-1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
Confidence
Confidence
(b)
Figure A4: (a) Illustration of Max Square (MS) loss and (b) Charbonnier penalty with different r)- Similar to entropy in Figure [ both the losses have vanishing gradient for high confidence predictions (confidences greater than 0.95).
21


Reviews:
[{"review_id": "xOPSHy5-0V", "reviewer": "Reviewer_nGh4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n\nThe paper introduces an information maximization loss for the test-time BN adaptation on unlabeled target mini-batch data.  The method has been well-motivated by pointing out the limitations in SOTA methods. \n- The proposed formulation incorporates the diversity regularizer to avoid the trivial collapsed solutions on mini-batch data.\n- It presents the analysis of the gradients of different losses for confidence maximization. And, proposes to use the negative log-likelihood ratio loss [Yao 2020] for TTA  which has non-vanishing gradients for high confidence predictions.  \n- It proposes to jointly update the IT module and BN parameters for TTA\n- The paper provides comprehensive experimental results in the paper and appendix. \n\nWeakness:\n- The proposed diversity loss is not very effective on mini-batch data. The results show the improvement is very marginal. \n- The paper should provide examples of failure cases,  and more explanation & discussion about the issues in the last paragraph of section 3.  \n- The novelty of the paper is relatively limited. IM loss, KL based $L_{div}$,  IT module, and the negative log-likelihood ratio are all proposed in the previous works. ", "summary_of_the_paper": "The paper proposes a new loss to improve test-time BN adaptation for domain adaptation. The proposed loss consists of two components: the diversity maximization loss and the confidence maximization loss. Specifically, they use a running estimate for the diversity loss based on KL divergence. They propose the hard and soft likelihood ratio for the confidence loss which has large gradients for high confidence predictions.   ", "summary_of_the_review": "Overall, the paper presents an information maximization (IM) loss for TTA on unlabeled target data. My main concerns are that the effectiveness of the proposed $L_{div}$ on mini-batch data, and the scale-normalization problem of the proposed logits. Hopefully, the authors can address my concern in the rebuttal period. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "uRCMEcJYT3c", "reviewer": "Reviewer_8M4e", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n\n– The paper is mostly well written and easy to follow\n\n– The proposed approach is intuitive, appears effective, and consistently outperforms competing methods for test-time adaptation\n\n– The paper includes a comprehensive set of experiments\n\n– The paper does well to compare with existing entropy minimization alternatives like MaxSquares and Charbonnier penalty. I would recommend including those results and a more detailed discussion in the main paper rather than appendix.\n\nWeaknesses \n\n– The proposed approach is largely a combination of existing methods – TENT (Wang et al., ICLR 2021), negative log-likelihood ratio loss (Yao et al., 2020), and batch-level diversity regularization (Li et al., arXiv 2020, Prabhu et al., ICCV 2021 [A]), for the test-time adaptation setting.\n\n[A] Prabhu, Viraj, et al. \"Sentry: Selective entropy optimization via committee consistency for unsupervised domain adaptation.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.\n\n– The paper lacks a proper ablation study: while the input transformation module is ablated, what is the individual contribution of each proposed piece?\n\n– The proposed loss does appear to have certain limitations, as it is unbounded and relies on proper scaling via model design (eg. batch norm layers) to prevent logit explosion.\n\n– “the soft likelihood ratio loss creates lower amplitude gradients for low-confidence self-supervision”: this does not appear to match Figure 1 (right), where SLR is slightly larger than HLR for low confidence (<0.2). Further, both SLR and HLR actually appear to have large gradients in this confidence regime as compared to hard pseudolabels – is this not problematic, since that would effectively upweight very low confidence predictions?\n\n\n----post-rebuttal----\n\nThe author response had addressed my concerns about the behavior of the proposed loss. In light of the paper's empirical contributions but limited technical novelty, I recommend a marginal accept. ", "summary_of_the_paper": "Studies the problem of test-time adaptation across distribution shift, and proposes i) a new self-training loss with better stability than entropy minimization ii) using a diversity regularizer and iii) an additional “input transformation” module. The approach is found to lead to improved performance on standard test-time adaptation settings.\n", "summary_of_the_review": "Interesting paper on improving test-time adaptation but I have some concerns (see weaknesses). I will be willing to reconsider my rating based on the author response.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "d4EXG_JsLTC", "reviewer": "Reviewer_e5PY", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n---\nThe self-supervised log likelihood ratio objective appears novel, as far as I am aware. And, for this problem setting, the combination of the aforementioned techniques is novel and leads to stronger empirical results than what has been previously reported.\n\nThe experiments are generally comprehensive and cover, as far as I can see, the important aspects of evaluation. I appreciate the results presented for both the \"offline\" and \"online\" adaptation settings, as well as the results adding all of the various techniques to Tent to evaluate whether any technique is of paramount importance.\n\nThe paper is generally well written and structured.\n\n\nWeaknesses\n---\nI think that the paper has improved on this point, but the motivation behind the general approach is still somewhat shaky. The idea that the model should extract a self-supervised learning signal from data points it is already very confident for still seems strange to me. Imagine if the model was already very confident for the entire batch of data points, but there is a (predicted) class imbalance in the batch. Would it not be the case that the model would adapt in this case when using the proposed approach, even though it would make more sense to not adapt at all, which for example entropy minimization would (roughly) do? And it is in general just unclear to me how incorporating a stronger gradient signal from confident points would help when it comes to ambiguous points. Perhaps what could be useful here is to actually \"show this in action\", e.g., take a real batch of data during adaptation and demonstrate how the model adapts with the proposed approach vs with Tent. This may provide greater intuition as to why the proposed approach is a good idea.\n\nNegative results are also of interest to the community, and to this end, including results on challenging distribution shift benchmarks such as ImageNet-A and ImageNet-v2, which prior work [1] has shown adaptation to be unhelpful for, would be great. Even just in the appendix, it would still be appreciated.\n\nA final minor nit from my previous review: I would still like to know whether or not a confidence of 0.82 is \"low\" for the corrupted image datasets or other instances of test distribution shift.\n\n[1] Schneider et al, \"Improving robustness against common corruptions by covariate shift adaptation\". NeurIPS 2020.", "summary_of_the_paper": "In the spirit of full disclosure: I have recently reviewed this paper, and several parts of my previous review are still applicable, thus I am copying in these parts when appropriate.\n\nThis paper presents a method for test time adaptation based on several techniques. These include a self-supervised adaptation objective based on log likelihood ratios, an additional regularizing objective to encourage diverse predictions, and an input transformation module that is also trained with the aforementioned objectives. Together, these techniques lead to better performance on ImageNet-C and ImageNet-R compared to Tent, a recent and similar test time adaptation method based on entropy minimization.", "summary_of_the_review": "Primarily due to my concerns above, I am initially recommending a weak accept of this paper. I am happy to engage in discussion with the authors and other reviewers in order to reach a more confident final recommendation.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "AF-313aqG4a", "reviewer": "Reviewer_9MXJ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n+ The joint adaptation of input transformations and model parameters is novel for test-time adaptation. Neither TTT (Sun et al. 2020), test-time normalization (Scheider et al. 2020), nor TENT (Wang et al. 2021) adapt the input. The related work section covers prior projects that learn to transform the input during training rather than testing.\n+ The proposed techniques—the alternative losses, the input transformation model, and diversity regularizer—improve accuracy jointly and separately. For instance, the input transformation model helps the proposed approach as well as TENT, and the diversity regularizer likewise fixes cases where TENT can fail. While these are not wholly new, it is still informative to double-check their effectiveness, ablate their combination, and show small but consistent improvement across multiple architectures and datasets (ImageNet-C, ImageNet-R, and the digit datasets MNIST/MNIST-M/USPS). \n+ The proposed extensions are still online and efficient, so this method seemingly could be deployed as easily as TENT. The only counter to this is that small accuracy drops are reported on unshifted data, where these drops are not seen (TENT, BN) or smaller (TTT) for other test-time methods.\n+ The results for test-time adaptation by optimization (TENT, HLR/SLR) on ImageNet-R are empirically new. Only test-time normalization (Schneider et al. 2020) had reported test-time results on this data.\n+ Design choices are justified with experimental results (main tables), visualization (Figure 1 for losses and gradients), and toy experiments (Appendix A.1).\n\nWeaknesses\n- The novelty of the proposed extensions is diminished by intersection with prior methods. The losses for HLR and SLR are from Yao et al. 2020, although this work is the first to bring them to fully test-time adaptation, and that is worthwhile. (For comparison, note that TENT argued for entropy which is obviously well-established as a loss). The diversity regularizer is a core part of SHOT (Liang et al. 2020), although this work calculates it differently with a moving average. The input transformation model is the most new, as input adaptation of this kind has not been done during testing, but there are close connections to training like ANT (Rusak et al.) and CyCADA (Tzeng et al.).\n- The proposed losses HLR and SLR further restrict the choice of parameters to adapt, as these losses would otherwise cause the predicted logits to grow without bound. Deriving an alternative loss without this restriction would be better to simplify the application of the method.\n- The amount of improvement on shifted data is marginal, at 2-3 points absolute in many cases, while there is harm on the unshifted/standard/clean data. This argues against the motivation for non-saturating losses. This is admitted on pg. 9 under \"Clean Images\" but is not remedied. A new loss or combination of losses that improves shifted accuracy without hurting clean accuracy would be more significant.\n- Input transformation (Sec. 3.1) is not closely studied or ablated. For instance, what if only the channel-wise input changes are included without the network? Do shallower networks do better or worse? Does input transformation help on all corruptions, or can it hurt?\n\nFor Rebuttal\n- Please provide a control experiment for the choice of optimization. How do TENT and TENT+ fare when optimized with Adam, the same solver as HLR/SLR? For TENT/TENT+, does raising the learning rate for SGD or Adam improve results by counteracting the saturation of the entropy loss? The need for non-saturating losses is a key claim so this is worth double-checking.\n- Please provide the results for TENT/TENT+ without freezing the weights of the deeper layers. The need for this freezing is a limitation of HLR and SLR, not TENT, so it is worth knowing if these additional parameters help the baselines.\n- Please comment on the learned transformation models, and in particular the learned transformation weight tau. How much do the scale gamma and shift beta help on their own? As a more novel part of this work, the input transformation module deserves more analysis.\n- Please explain the data subset experiments in more detail (Figure 3). Why does TENT fail as the split fraction reaches 1.0? In the TENT paper, there are generalization results with adaptation on target train and evaluation on target test, and the method still helps in that case. What is the justification for these use cases? Would it not be better to always adapt on all the data that is encountered?\n\nMiscellaneous Feedback\n- [clarity] please summarize results with the mean where appropriate, for instance by including the mean over corruption types in Table 1\n- [clarity] consider including qualitative results of the learned image transformations, for instance in the appendix, to show the types and degrees of transformation.\n- [text] in the related work, change \"domain adaptation train\" to \"domain adaptation methods train\"\n- [text] in the related work, change \"such setting refrain the cost\" to \"such settings spare the cost\"\n- [text] in Sec. 3.2.1, change \"One option are\" to \"One option is\"", "summary_of_the_paper": "Test-time adaptation by entropy minimization can help models adapt to dataset shifts like corruptions without altering training. This work extends tent, an entropy minimization method, by proposing alternative non-saturating losses, adding a diversity regularizer, and adapting the input data along with the model parameters. The input is adapted by applying a convolutional image transformation model between the input and classification model. These extensions do not need more optimization iterations or supervision than the baselines: the method adapts online and efficiently without auxiliary supervision. Experiments on the corruption benchmark ImageNet-C and the newer benchmark ImageNet-R report reduced generalization error. The improvements are there but marginal, and they are consistent across multiple baseline architectures (ResNet, DenseNet, MobileNet, etc.). However, the clean accuracy reduced, so the proposed method does not strictly dominate prior work.", "summary_of_the_review": "This work makes a reasonable but minor contribution that can inform further extensions of test-time adaptation. A large part of this work is the double-checking of elements of test-time adaptation methods, with only marginal empirical novelty and significance, or the importing of techniques from other scopes, with either no or only marginal technical novelty and significance. It is a pity that the most new part, the test-time input transformer, is not further studied and improved to give this work a more independent dimension of contribution. At the same time, the harm to accuracy on normal data is cause for hesitation to accept the proposed changes. While there is value in this work, rejection is recommended so that (1) the input transformer can be more fully covered and (2) the issue of improving out-of-distribution accuracy at the cost of in-distribution accuracy can be resolved.\n\n**Final Review** The rebuttal thoroughly clarified results and offered additional experiments to empirically justify the contributions and reduce worries about potential issues. In particular, the results with the baselines of TENT/TENT+ now make sense with the clarification of online/offline results (please underline this in the paper), and the proposed method is not so sensitive to the choice of frozen layers. I have raised my score to 6, and I would have considered a 7 if there were such a rating. I did not go higher because the trade-off of lower accuracy on unshifted data for higher accuracy on shifted data remains. Nevertheless there is informative material here, in the main paper and appendix, and so I vote for acceptance so that this work can inform the burgeoning direction of test-time adaptation. Along with the more novel parts, such as input transformation during testing, this work also helpfully confirms and tunes other parts like the diversity regularizer in ways that future work can simply adopt.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper considers test time adaptation to distribution shift which is a very important and impactful problem. The authors propose an empirical method that has different pieces, the most important ones being input transformation and confidence maximization and using likelihood ratio loss.

There were various concerns that got addressed during the rebuttal period such as, novelty of the proposed method, ablation study of different parts of the model, novelty and importance of diversity regularizer, choice of optimization. However there are still three remaining concerns that addressing them will improve the paper significantly: First, clear motivation behind the method for the cases when the model is certain but we have data imbalance. Second, analysis in the online setting of batch-by-batch prediction and adaptation. Third,
establishing the claim regarding data subset experiment that it enable the model to adapt on a subset of data and later switch to complete execution mode without adaptation for efficient run time and improved throughput. How is the method to know the data distribution has changed, or that it has sufficiently adapted to it when the data distribution is not changing?
Decision: Reject