T ENT : F ULLY T EST -T IME A DAPTATION BY E NTROPY M INIMIZATION
Anonymous authors
Paper under double-blind review
A BSTRACT
A model must adapt itself to generalize to new and different data during testing.
This is the setting of fully test-time adaptation given only unlabeled test data and the model parameters.
We propose test-time entropy minimization (tent): we optimize for model conﬁdence as measured by the entropy of its predictions.
During testing, we adapt the model features by estimating normalization statistics and optimizing channel-wise afﬁne transformations.
Tent improves robustness to corruptions for image classiﬁcation on ImageNet and CIFAR-10/100 and achieves state-of-the-art error on ImageNet-C for ResNet-50. Tent shows the feasibility of target-only domain adaptation for digit classiﬁcation from SVHN to MNIST/MNIST-M/USPS and semantic segmentation from GTA to Cityscapes.
I NTRODUCTION
Deep networks can achieve high accuracy on training and testing data from the same distribution, as evidenced by tremendous benchmark progress (Krizhevsky et al., 2012; Simonyan & Zisserman, 2015;
He et al., 2016).
However, generalization to new and different data is limited (Hendrycks &
Dietterich, 2019; Recht et al., 2019; Geirhos et al., 2018).
Accuracy suffers when the training (source) data differ from the testing (target) data, a condition known as dataset shift (Quionero-Candela et al., 2009).
Models can be sensitive to shifts during testing that were not known during training, whether natural variations or corruptions, such as unexpected weather or sensor degradation.
Nevertheless, it can be necessary to deploy a model on different data distributions, so adaptation is needed.
During testing, the model must adapt given only its parameters and the target data.
This fully test-time adaptation setting cannot rely on source data or supervision.
Neither is practical when the model ﬁrst encounters new testing data, before it can be collected and annotated, as inference must go on.
Real-world usage motivates fully test-time adaptation by data, computation, and task needs:
Availability.
A model might be distributed without source data for bandwidth, privacy, or proﬁt.
Efﬁciency.
It might not be computationally practical to (re-)process source data during testing.
Accuracy.
A model might be too inaccurate without adaptation to serve its purpose.
We take the entropy of model predictions during testing as our adaptation objective.
We call this the test entropy and name our method tent after it.
Entropy is related to error, as more conﬁdent predictions are all-in-all more correct (Figure 1).
On corrupted data, entropy is an indicator of the severity of the corruption, with a strong rank correlation to the loss for image classiﬁcation (Figure 2).
To minimize entropy, tent normalizes and transforms the model features on target data by estimating statistics and optimizing afﬁne parameters.
This choice of low-dimensional, channel-wise feature modulation is efﬁcient to update during testing, even for online adaptation.
Tent does not restrict or alter model training: it is independent of the source data given the model parameters.
If the model can be run, it can be adapted.
Most importantly, tent effectively reduces not just entropy but error.
Our results evaluate robustness to common image corruptions and accuracy under domain shift for digit recognition.
For reference results with more data and optimization, we evaluate methods for robust training, domain adaptation, and self-supervised learning given the labeled training data.
Tent achieves less error given only the test data.
Our analysis supports entropy as an objective, backs the generality of tent across architectures, and varies the amount of parameters and data for adaptation.
[Figure] [Figure]
Our contributions
• We highlight the setting of fully test-time adaptation with only target data and no source data.
We suggest benchmarking adaptation with ofﬂine and online use of target data.
• We examine entropy as an adaptation objective, and propose tent: a test-time entropy minimization scheme to reduce generalization error by reducing the entropy of model predictions.
• For robustness to corruptions, tent reaches 44 . 0% error on ImageNet-C, better than the state-ofthe-art for robust training ( 49 . 6% ) and the strong baseline of test-time normalization ( 51 .
• For domain adaptation, tent is capable of target-only adaptation for digit classiﬁcation and semantic segmentation, and even rivals methods that use source data and more optimization on SVHN → MNIST/USPS.
S ETTING : F ULLY T EST -T IME A DAPTATION
Adaptation addresses generalization from source to target.
A model f θ ( x ) with parameters θ trained on source data and labels x s , y s may not generalize when tested on shifted target data x t .
Tablesummarizes adaptation settings, their required data, and types of losses.
Our fully test-time adaptation setting uniquely requires only the model f θ and unlabeled target data x t .
Existing adaptation settings extend training given more data and supervision.
Transfer learning by ﬁne-tuning (Donahue et al., 2014;
Yosinski et al., 2014) needs target labels to (re-)train with a supervised loss L ( x t , y t ) .
Without target labels, our setting denies this supervised training.
Domain adaptation (DA) (Quionero-Candela et al., 2009; Saenko et al., 2010; Ganin & Lempitsky, 2015;
Tzeng et al., 2015) needs both the source and target data to train with a cross-domain loss L ( x s , x t ) .
Test-time training (TTT) (Sun et al., 2019b) needs the source data to jointly train with a supervised loss and an unsupervised loss L ( x s ) .
Without source, our setting denies source supervision L ( x s , y s ) for joint training across domains (DA) or losses (TTT).
These settings have their purposes, but do not cover all practical cases when source, target, or supervision are not simultaneously available.
Unexpected target data during testing requires test-time adaptation.
TTT and our setting adapt the model by optimizing an unsupervised loss during testing L ( x t ) .
During training, TTT jointly optimizes this same loss on source data L ( x s ) with a supervised loss L ( x s , y s ) , to ensure the parameters θ are shared across losses for compatibility with adaptation by L ( x t ) .
Fully test-time adaptation is independent of the training data and training loss given the parameters θ .
By not changing training, our setting has the potential to require less data and computation for adaptation.
M ETHOD : T EST E NTROPY M INIMIZATION VIA F EATURE M ODULATION
We adapt the model through test-time optimization to minimize the entropy of its predictions by modulating its features.
We call this adaptation method tent , for test entropy.
Tent requires a compatible model, an objective to minimize (Section 3.1), and parameters to optimize over (Section 3.2) to fully deﬁne the algorithm (Section Section 3.3).
Figure 3 outlines our fully test-time adaptation method.
[Table]
+JUM95msYz1fUANl0LxNgqU/D7RnEaB5J1gfDX1O09cGxGrW8wS7kd0qEQoGEUrPfRGFPNs8oj9as2tuzOQZeIVpAYFWv3qV28QszTiCpmkxnQ9N0E/pxoFk3xS6aWGJ5SN6ZB3LVU04sbPZwdPyIlVBiSMtS2FZKb+nshpZEwWBbYzojgyi95U/M/rphhe+LlQSYpcsfmiMJUEYzL9ngyE5gxlZgllWthbCRtRTRnajCo2BG/x5WXSOat7jbrn3TRqzcsijzIcwTGcggfn0IRraEEbGETwDK/w5mjnxXl3PuatJaeYOYQ/cD5/ANn4kOk=</latexit>
ˆ y t
Compatibility The model must be trained for the supervised task, probabilistic, and differentiable.
No supervision is provided during testing, so the model must already be trained.
Measuring the entropy of predictions requires a distribution over predictions, so the model must be probabilistic.
Gradients are required for fast iterative optimization, so the model must be differentiable.
Choice of Model and Task We choose image classiﬁers as representative models for supervised learning with deep networks.
These models are probabilistic by their prediction of softmax class distributions and end-to-end differentiable by their design.
3. E NTROPY O BJECTIVE
Our test-time objective L ( x t ) is the entropy H (ˆ y ) of the model prediction during testing ˆ y =
f θ ( x t ) .
In particular, we measure the Shannon entropy (Shannon, 1948), H (ˆ y ) =
− (cid:80) c
p (ˆ y c ) log p (ˆ y c ) for the probability ˆ y c of class c .
As a measure of the task predictions, H (ˆ y =
f θ ( x )) is a function of the model parameters, and therefore the supervised training.
In this way entropy is task-general, as it is deﬁned for any probabilistic task, but at the same time task-speciﬁc, as it varies with the task training.
In contrast, proxy tasks for self-supervised learning are not directly related to the supervised task.
Proxy tasks derive a self-supervised label y (cid:48) from the input x t without the task label y .
For concreteness, examples of these proxies include rotation prediction (Gidaris et al., 2018), context prediction (Doersch et al., 2015), and cross-channel auto-encoding (Zhang et al., 2017).
Too much progress on a proxy task could interfere with performance on the supervised task, and self-supervised adaptation methods have to limit or mix updates accordingly (Sun et al., 2019b;a).
As such, care is needed to choose a proxy compatible with the domain and task, to design the architecture for the proxy model, and to balance optimization between the task and proxy objectives.
Our test entropy objective does not present these difﬁculties.
Entropy is an unsupervised objective that depends only on ˆ y .
No further effort or choice is needed to adopt it.
On the other hand, supervised learning and self-supervised learning take more effort in annotation or design choices.
3. M ODULATION P ARAMETERS
The model parameters θ are a natural choice for test-time optimization, and these are the choice of prior work for train-time entropy minimization in semi-supervised (Grandvalet & Bengio, 2005), few-shot (Dhillon et al., 2020), and domain adaptation (Carlucci et al., 2017) regimes.
However, θ is the only representation of the training/source data in our setting, and altering θ could cause the model to drift from the training task.
Furthermore, f can be nonlinear and θ can be high dimensional, making optimization too sensitive and inefﬁcient for test-time usage.
For stability and efﬁciency, we instead only update feature modulations that are linear (scales and shifts), and low-dimensional (channel-wise).
Figure 4 shows the two steps of our modulations: normalization by statistics and transformation by parameters.
Normalization centers and standardizes
[Table]
>   + @
H /@,   + @ H /@
[Figure]
For implementation the normalization
3. A LGORITHM
Initialization
The optimizer collects the afﬁne transformation parameters { γ l,k , β l,k } for each normalization layer l and channel k in the source model.
The remaining parameters θ \ { γ l,k , β l,k } are ﬁxed.
The normalization statistics { µ l,k , σ l,k } from the source data are discarded.
Iteration Each step updates the normalization statistics and transformation parameters on a batch of test data.
The normalization statistics are updated by moving averages on the test data for each layer in turn, during the forward pass.
The transformation parameters γ, β are updated by the gradient of the prediction entropy ∇ H (ˆ y ) , during the backward pass.
For online adaptation, forward is repeated once the model is updated, to improve inference for every test point.
This only needs 2 × the inference time plus 1 × the gradient time per test point vs. the 1 × inference time of the unadapted model.
Termination For online adaptation, no termination is necessary, and iteration continues as long as there is test data.
For ofﬂine adaptation, we update for a single epoch, as it is simple and efﬁcient.
Inference is then repeated for the whole test set.
Of course, it is possible to extend adaptation by continuing to optimize for multiple epochs.
E XPERIMENTS
We evaluate tent for corruption robustness on CIFAR-10/CIFAR-100 and ImageNet, and for domain adaptation on digit adaptation from SVHN to MNIST/MNIST-M/USPS.
Our implementation is in PyTorch (Paszke et al., 2019) with the the pycls library (Radosavovic et al., 2019).
Our (anonymized) code is included with this submission, and the code will be released for publication.
Datasets We run on image classiﬁcation datasets for corruption and domain adaptation conditions.
For large-scale experiments we choose ImageNet (Russakovsky et al., 2015), with 1,000 classes, a training set of 1.2 million, and a validation set of 50,000.
For experiments at an accessible scale we choose CIFAR-10/CIFAR-100 (Krizhevsky, 2009), with 10/100 classes, a training set of 50,000, and a test set of 10,000.
For domain adaptation we choose SVHN (Netzer et al., 2011) as source and
MNIST (LeCun et al., 1998)/MNIST-M (Ganin & Lempitsky, 2015)/USPS (Hull, 1994) as targets, with ten classes for the digits 0–9.
SVHN has color images of house numbers from street views with a training set of 73,257 and test set of 26,032.
MNIST/MNIST-M/USPS have handwritten digits with a training sets of 60,000/60,000/7,291 and test sets of 10,000/10,000/2,007.
Models For corruption we use residual networks (He et al., 2016) with 26 layers (R-26) on CIFAR10/100 and 50 layers (R-50) on ImageNet.
For domain adaptation we use the R-26 architecture.
For fair comparison, all methods in each experimental condition share the same architecture.
Our networks are equipped with batch normalization (Ioffe & Szegedy, 2015).
For the source model without adaptation, the normalization statistics are estimated during training on the source data.
For all test-time adaptation methods, we estimate these
[Table]
[Figure]
Tent improves Optimization We optimize the modulation parameters γ, β following the training hyperparameters for the source model with few changes.
On ImageNet we optimize by SGD with momentum, while on other datasets we optimize by Adam (Kingma & Ba, 2015).
We lower the batch size (BS) to reduce memory usage, then lower the learning rate (LR) by the same factor to compensate (Goyal et al., 2017).
On ImageNet, we set BS = 64 and LR = 0.00025, and on other datasets we set BS =and LR = 0.001.We shufﬂe the test data to avoid ordering effects, and control for shufﬂing by sharing the order across methods.
Baselines We compare to domain adaptation, self-supervision, normalization, and pseudo-labeling:
• source applies the trained classiﬁer to the test data without adaptation, • adversarial domain adaptation (RG) reverses the gradients of a domain classiﬁer on source and target to optimize for a domain-invariant representation (Ganin & Lempitsky, 2015), • self-supervised domain adaptation (UDA-SS) jointly trains self-supervised rotation and position tasks on source and target to optimize for a shared representation (Sun et al., 2019a), • test-time training (TTT) jointly trains for supervised and self-supervised tasks on source, then keeps training the self-supervised task on target during testing (Sun et al., 2019b), • test-time normalization (BN) updates batch normalization statistics (Ioffe & Szegedy, 2015) on the target data during testing (Schneider et al., 2020; Nado et al., 2020), • pseudo-labeling (PL) tunes a conﬁdence threshold, assigns predictions over the threshold as labels, and then optimizes the model to these pseudo-labels before testing (Lee, 2013).
Only test-time normalization (BN), pseudo-labeling (PL), and tent (ours) are fully test-time adaptation methods.
See Section 2 for an explanation and contrast with domain adaptation and test-time training.
4. R OBUSTNESS TO C ORRUPTIONS
To benchmark robustness to corruption, we make use of common image corruptions (See appendix
Section A for examples).
The CIFAR-10/100 and ImageNet datasets are turned into the CIFAR10/100-C and ImageNet-C corruption benchmarks by duplicating their test/validation sets and applying 15 types of corruptions at ﬁve severity levels (Hendrycks & Dietterich, 2019).
Tent improves error more with less data and computation.
Table 2 reports errors averaged over corruption types at the severest level of corruption.
On CIFAR-10/100-C we compare all methods, including those that require joint training across domains or losses, given the convenient size of this dataset.
Adaptation is ofﬂine for fair comparison across this mixture of methods.
Tent improves on the fully test-time adaptation baselines (BN, PL) but also the domain adaptation (RG, UDA-SS) and test-time training (TTT) methods that need the source data and several epochs of optimization.
Tent consistently improves error across corruption types.
Figure 5 plots the errors for each corruption type averaged over levels on ImageNet-C.
We compare the most efﬁcient methods— source, normalization, and tent—given the large-scale size of the source data (>1 million images) that other methods rely on and the 75 target combinations of corruption types and levels.
Tent and BN adapt online to rival the efﬁciency of inference without adaptation.
Note that many conditions have
[Table]
over 50% error even after adaptation by normalization, but tent nevertheless further reduces the error.
Tent reaches the least error for all corruptions types without harming the error on the original test set.
Tent reaches new state-of-the-art without extending training.
The state-of-the-art methods for robustness extend training with adversarial noise (ANT) (Rusak et al., 2020) for 50 . 2% error or mixtures of data augmentations (AugMix) (Hendrycks et al., 2020) for 51 . 7% error.
Combined with stylization from external images (SIN) (Geirhos et al., 2019), ANT+SIN reaches 47 .
Tent reaches a new state-of-the-art of 44 . 0% by online adaptation and 42 .
3% by ofﬂine adaptation. It improves on
ANT for all types except noise, on which ANT is trained.
This requires just one gradient per test point, without further optimization on the training set, much less external images.
Among fully test-time adaptation methods, tent reduces the error beyond test-time normalization for 18% relative improvement.
In concurrent work, Schneider et al.
(2020) report 49 .
3% error for test-time normalization, for which tent still gives 14% relative improvement.
4. T ARGET -O NLY D OMAIN A DAPTATION
We adopt the established setting of digit adaptation (Ganin & Lempitsky, 2015; Tzeng et al., 2015; 2017; Shu et al., 2018) with adaptation from SVHN to MNIST/MNIST-M/USPS.
Recall that unsupervised domain adaptation makes simultaneous use the labeled source data and unlabeled target data, while our fully test-time adaptation setting denies use of source data.
Tent adapts to target without source.
Table 3 reports the target errors for domain adaptation and fully test-time adaptation methods.
Test-time normalization (BN) marginally improves, while adversarial domain adaptation (RG) and self-supervised domain adaptation (UDA-SS) improve more by joint training on source and target.
Tent always has lower error than the source model and BN, and it achieves the lowest error in 2/3 cases, even in just one epoch and without use of source data.
While encouraging for fully test-time adaptation, unsupervised domain adaptation remains necessary for the highest accuracy and harder shifts.
For SVHN-to-MNIST, DIRT-T (Shu et al., 2018) achieves a remarkable 0 .
6% error 1 .
For MNIST-to-SVHN, a difﬁcult shift with source-only error of 71 . 3% ,
DIRT-T reaches 45 .
5% and UDA-SS reaches 38 . Tent fails on this shift and increases error to 79 . Success in these regimes presently requires joint optimization over source and target. Tent needs less computation, but still improves with more.
Tent reduces computation by (1) only using target data and (2) only taking one gradient per point.
RG & UDA-SS use the source data (SVHN train), which is ∼ 7 × the size of the target data (MNIST test), and optimize for 10 epochs.
Tent adapts with ∼ 70 × less computation.
When tent is adapted for more epochs, its error improves further while still only using one-seventh the amount of data.
Tent scales to semantic segmentation.
To show scalability to large models and inputs, we evaluate semantic segmentation (pixel-wise classiﬁcation) on a domain shift from a simulated source to a real target.
The source is GTA (Richter et al., 2017), a video game in an urban environment, and the target is Cityscapes (Cordts et al., 2016), an urban autonomous driving dataset.
The model is HRNet-W18, a fully convolutional network (Shelhamer et al., 2017) with high-resolution architecture (Wang et al., model [Figure]
The target intersection-over-union scores (higher is better) are source 28 . 8% , BN 31 .
4% , and tent 35 .
8% with ofﬂine optimization by Adam.
For adaptation to a single image , tent reaches 36 . 4% in with episodic optimization. See the appendix for a qualitative example (Section B).
4. A NALYSIS
Tent reduces entropy and error.
Figure 6 veriﬁes tent does indeed reduce the entropy and the task loss (softmax cross-entropy).
We plot changes in entropy and loss on CIFAR-100-C for all 75 corruption type/level combinations.
Both axes are normalized by the maximum entropy of a prediction ( log 100 ) and clipped to ± 1 .
Most points have lower entropy and error after adaptation.
Tent needs feature modulation.
We ablate the normalization and transformation steps of feature modulation.
Not updating normalization increases errors, and can fail to improve over BN and PL.
Not updating transformation parameters reduces the method to test-time normalization.
Updating only the last layer of the model can improve but then degrades with further optimization.
Updating the full model parameters θ never improves over the unadapted source model.
Tent modulation generalizes across target data. As tent simultaneously adapts and tests on target, its adaptation could be limited to these speciﬁc data points. We decouple speciﬁc adaptation to each point and general adaptation to the domain by adapting tent to target train rather than target test. Test errors drop: CIFAR-100-C error goes from 37.3% to 34.2% and SVHN-to-MNIST error goes from 8.2% to 6.5%.
(Train is larger than test, so more data is resulting in less error.)
The adapted modulation is not point-speciﬁc, but domain-general, as adapting on train helps predicting on test.
Tent modulation differs from normalization.
Modulation normalizes and transforms, but what is the combined effect on features?
Figure 7 contrasts adapted features on corrupted data against reference features on uncorrupted data.
We examine features from the source model, normalization, tent, and an oracle that optimizes on the target labels.
Normalization adapts features to be more like the reference, but tent does not.
Instead, tent adapts features to be more like the oracle.
This suggests a different, task-speciﬁc effect of test entropy minimization.
See the appendix for feature visualizations of more layers (Section C).
Tent adapts alternative architectures In principle, tent is architecture agnostic.
To gauge its generality, we evaluate new architectures based on self-attention (SAN)
(Zhao et al., 2020) and equilibrium solving (MDEQ)
(Bai et al., 2020) for corruption robustness on CIFAR-100-C. Tablereports the errors for the source model, BN adaptation, and tent adaptation.
Tent reduces their error with the same settings as convolutional residual networks, in spite of their distinct architectures.
[Table]
R ELATED W ORK
We relate tent to existing adaptation, entropy minimization, and feature modulation methods.
Train-Time Adaptation Domain adaptation methods train a joint model of the source and target by cross-domain losses L ( x s , x t ) .
These losses optimize feature alignment (Gretton et al., 2009; Sun et al., 2017), adversarial invariance (Ganin & Lempitsky, 2015; Tzeng et al., 2017), or shared proxy tasks (Sun et al., 2019a).
While they are effective in their setting, they do not apply when joint use of source and target is denied.
Tent adapts entirely to target without joint modeling of source.
Recent “source-free” methods (Li et al., 2020; Kundu et al., 2020) also adapt without source data.
Both rely on generative modeling and optimize multiple models with multiple losses. Kundu et al.
(2020) also alter training.
Tent does not need generative modeling, nor does it alter training, making it much more computationally efﬁcient and capable of online adaptation.
We encourage the reader to examine each of these works on the frontier of adaptation without source. To the best of our knowledge, Chidlovskii et al. (2016) are the ﬁrst to motivate adaptation without source data due to legal or commercial concerns. They adapt predictions, rather than models, by applying denoising auto-encoders to the output.
As such their method and experimental scope differ signiﬁcantly from our end-to-end and online adaptation method. Test-
Time Adaptation Tent adapts by test-time optimization and normalization.
Test-time training (TTT) (Sun et al., 2019b) also optimizes during testing, but differs in its loss and alters training.
TTT’s loss relies on a proxy task, such as recognizing rotations of an image, and therefore depends on the choice of proxy.
(Indeed, its authors caution that the proxy must be “both well-deﬁned and non-trivial in the new domain”).
Our test entropy loss is measured without any proxy task.
Furthermore, TTT must alter training to include its proxy loss.
Tent does not alter training, and so it applies to models alone withour their source data.
Normalizing feature statistics is common for domain adaptation (Gretton et al., 2009; Sun et al., 2017).
For batch normalization Li et al.
; Carlucci et al.
(2017) separate the source and target statistics during training.
In concurrent work, Schneider et al.
Nado et al. (2020) estimate target statistics during testing and show this improves generalization. Tent builds on test-time normalization to further reduce generalization error.
Entropy Minimization Entropy minimization is a key regularizer for domain adaptation (Carlucci et al., 2017; Shu et al., 2018; Saito et al., 2019; Roy et al., 2019), semi-supervised learning (Grandvalet & Bengio, 2005; Lee, 2013; Berthelot et al., 2019), and few-shot learning (Dhillon et al., 2020).
Regularizing entropy penalizes decision boundaries at high densities in the data distribution to thereby improve accuracy for distinct classes (Grandvalet & Bengio, 2005).
These methods all regularize entropy during training in concert with other supervised and unsupervised losses on additional data.
Tent is the ﬁrst method to minimize entropy during testing, for adaptation to corruption and domain shift, without other losses or data.
Entropic losses are common; our contribution is to exhibit the effectiveness of entropy as the sole loss for fully test-time adaptation.
Feature Modulation Modulation adjusts a model so that it varies with its input.
Following signal processing usage, the modulation is usually simpler than the model, for example in its lower dimensionality.
We optimize feature modulations instead of the full model for stable and efﬁcient adaptation.
We choose modulation by channel-wise afﬁne transformation, for its effectiveness in tandem with normalization (Ioffe & Szegedy, 2015; Wu & He, 2018), and for its ﬂexibility to condition the model for different tasks (Perez et al., 2018).
These normalization and conditioning methods optimize the modulation during training by a supervised loss, but keep it ﬁxed during testing.
We optimize the modulation during testing by an unsupervised loss, so that it can adapt to different targets.
C ONCLUSION
Tent reduces generalization error for fully-test time adaptation by entropy minimization.
This is remarkable in that its entropy objective, while unsupervised, is deﬁned by the supervised model training.
In effect, it seems that the model has learned enough to supervise itself on shifted data.
While there are still gaps in accuracy on corruptions and different domains, and therefore more adaptation is needed, this is an encouraging step.
Improvements from tent show that a model knows more than it can infer in one go: it generalizes more by adapting to feedback from its own predictions.
Our fully test-time adaptation setting and experiments should encourage more exploration of what models may already know about the data distribution and how this can be turned into further self-improvement
A PPENDIX
We report supplementary results for the method as described in Section 3.
All of these results are collected with the same method as-is , with differences only in (1) the input and (2) the pre-trained model to be adapted.
A R OBUSTNESS TO C ORRUPTIONS
In Section 4.1 we evaluate methods on a common image corruptions benchmark.
In Table 2 we report results on the most severe level of corruption, level 5.
In this appendix we include examples of these image corruptions, and report results across corruption types and levels for completeness.
Example Corruptions We summarize the image corruption types for the benchmark in Figure 8.
Varying Severity Error rates for all corruption types and all levels 1–5 are reported in Tables 5, 6, 7, and 9.
The model and optimization details follow Section 4.1, and only the level of input corruption differs.
[Table] [Table]
[Table] [Table]
Gaussian Noise
Shot Noise
Impulse Noise
Defocus Blur Frosted Glass Blur
Motion Blur
Zoom Blur
Snow
Frost
Fog
Brightness
Contrast
Elastic
Pixelate
JPEG
[Table]
B T ARGET - ONLY D OMAIN A DAPTATION FOR S EMANTIC S EGMENTATION
We report a qualitative result on target-only domain adaptation for semantic segmentation (pixel-wise classiﬁca- tion) with a simulation-to-real (sim-to-real) domain shift.
For this sim-to-real condition, the source data is simulated while the target data is real.
Our source data is GTA Richter et al.
(2017), a visually-sophisticated video game set in an urban environment [Table] and our target data is
Cityscapes Cordts et al.
an urban autonomous driving dataset.
The supervised model is HRnet-W18, a [Figure] C F EATURE D ISTRIBUTIONS ACROSS L AYERS AND M ETHODS
[Figure] L a y e r L a y e r L a y e r
L a y e r L a y e r L a y e r
L a y e r L a y e r
L a y e r