Robust Continual Test-time Adaptation: Instance-aware BN and Prediction-balanced Memory
s
AfﬁliationAddress email
Abstract
Test-time adaptation (TTA) is an emerging paradigm that addresses distributionalshifts between training and testing phases without additional data acquisitionor labeling cost; only unlabeled test data streams are used for continual modeladaptation.
Previous TTA schemes assume that the test samples are independentand identically distributed (i.i.d.), even though they are often temporally correlated(non-i.i.d.) in application scenarios, e.g., autonomous driving.
We discover thatmost existing TTA methods fail dramatically under such scenarios.
Motivatedby this, we present a new test-time adaptation scheme that is robust against noni.i.d.
test data streams.
Our novelty is mainly two-fold: (a) Instance-Aware Batch
Normalization (IABN) that corrects normalization for out-of-distribution samples,and (b) Prediction-balanced Reservoir Sampling (PBRS) that simulates i.i.d.
datastream from non-i.i.d.
stream in a class-balanced manner.
Our evaluation withvarious datasets, including real-world non-i.i.d.
streams, demonstrates that theproposed robust TTA not only outperforms state-of-the-art TTA algorithms in thenon-i.i.d.
setting, but also achieves comparable performance to those algorithmsunder the i.i.d. assumption.
Introduction
While deep neural networks (DNNs) have been successful in several applications, their performancedegrades under distributional shifts between the training data and test data [26].
This distributionalshift hinders DNNs from being widely deployed in many risk-sensitive applications, such as autonomous driving, medical imaging, and mobile health care, where new types of test data unseenduring training could result in undesirable disasters.
For instance, Tesla Autopilot has caused 12“deaths” until recently [2].
To address this problem, test-time adaptation (TTA) aims to adapt DNNsto the target/unseen domain with only unlabeled test data streams, without any additional data acquisition or labeling cost.
Recent studies reported that TTA is a promising, practical direction to mitigatedistributional shifts [23, 27, 35, 4, 38].
Prior TTA studies typically assume (implicitly or explicitly) that a target test sample x t at time t andthe corresponding ground-truth label y t (unknown to the learner) are independent and identicallydistributed (i.i.d.) following a target domain, i.e., ( x t , y t ) is drawn independently from a time-invariantdistribution P T ( x , y ) .
However, the distribution of online test samples often changes across the timeaxis, i.e., ( x t , y t ) ∼ P T ( x , y | t ) in many applications; for instance, AI-powered self-driving car’sobject encounter will be dominated by cars while driving on the highway, but less dominated bythem on downtown where other classes such as pedestrians and bikes are visible.
In human activityrecognition, some activities last for a short term (e.g., a fall down), whereas certain activities last longer(e.g., a sleep).
Figure 1 illustrates that some data distributions in the real world, such as autonomousdriving and human activity recognition, are often temporally correlated.
Considering that most
… … … 𝑡𝑖𝑚𝑒 …
Car
Van Pedestrian
Person (sitting)
Cyclist
Misc
[Figure]
Standing
[Figure]
KKF PQPKKF      % N C UU K H K E C V K QP  G TT Q T  2GTHQTOCPEG1XGTXKGY 5QWTEG$05VCVU2.10&#6'06%Q66#
NOTE existing TTA algorithms simply use an incoming batch of test samples for adaptation [23, 27, 35, 38],the model might be biased towards these imbalanced samples under the temporally correlated teststreams.
Figure 2 compares the performance of the state-of-the-art TTA algorithms under the i.i.d.
andnon-i.i.d.
1 conditions.
While the TTA methods perform well under the i.i.d.
assumption, their errorsincrease under the non-i.i.d.
case.
Adapting to temporally correlated test data results in overﬁtting totemporal distributions, which in turn harms the generalization of the model.
Motivated by this, we present a NOn-i.i.d.
TEst-time adaptation scheme, NOTE , that consists oftwo components: (a) Instance-Aware Batch Normalization (IABN) and (b) Prediction-Balanced
Reservoir Sampling (PBRS).
First, we propose a novel normalization layer, IABN, that eliminatesthe dependence on temporally correlated data for adaptation while being robust to distributionshifts.
IABN detects out-of-distribution instances sample by sample and corrects via instance-awarenormalization.
The key idea of IABN is synthesizing Batch Normalization (BN) [14] with Instance
Normalization (IN) [31] in a unique way; it calculates how different the learned knowledge (BN) isfrom the current observation (IN) and corrects the normalization by the deviation between IN and BN.
Second, we present PBRS that resolves the problem of overﬁtting to non-i.i.d.
samples by mimickingi.i.d.
samples from non-i.i.d.
streams.
By utilizing predicted labels by the model, PBRS aims forboth time-uniform sampling and class-uniform sampling from the non-i.i.d.
streams, and stores the‘simulated’ i.i.d.
samples in memory.
With the i.i.d.-like batch in the memory, PBRS enables themodel to adapt to the target domain without being biased to temporal distributions.
We evaluate NOTE with state-of-the-art TTA baselines [23, 27, 18, 22, 4, 38] on multiple datasets,including common TTA benchmarks (CIFAR10-C and CIFAR100-C [11]) and real-world non-i.i.d.datasets (KITTI [8], HARTH [21], and ExtraSensory [32]).
Our results suggest that NOTE not onlysigniﬁcantly outperforms the baselines under non-i.i.d.
test data, e.g., it achieves a 21.1% error rateon CIFAR10-C which is on average 15.1% lower than the state-of-the-art method [4], but also showscomparable performance even under the i.i.d.
assumption, e.g., 17.6% error on CIFAR10-C where thebest baseline [38] achieves 17.8% error.
Our ablative study demonstrates the individual effectivenessof IABN and PBRS and further highlights their synergy when jointly used.
Finally, we summarize the key characteristics of NOTE.
First, NOTE is a batch-free inferencealgorithm (requiring a single instance for inference), different from the state-of-the-art TTA algorithms [23, 27, 35, 4, 38] where a batch of test data is necessary for inference to estimate normalizationstatistics (mean and variance).
Second, while some recent TTA methods leverage augmentationsto improve performance at the cost of additional forwarding passes [29, 38], NOTE requires onlya single forwarding pass.
NOTE updates only the normalization statistics and afﬁne parameters in
IABN, which is, e.g., approximately 0.02% of the total trainable parameters in ResNet18 [10].
Third,
NOTE’s additional memory overhead is negligible.
It merely stores predicted labels of test data to run
PBRS.
These characteristics make NOTE easy to apply to any existing AI system and particularly,are beneﬁcial in latency-sensitive tasks such as autonomous driving and human health monitoring.
Background2.1 Problem setting: test-time adaptation with non-i.i.d. streams
Test-time adaptation.
Let D S
= {X S , Y} be the data from the source domain and D T =
{X T , Y}be the data from the target domain to adapt to.
Each data instance and the corresponding ground-truthlabel pair ( x i , y i ) ∈
X S × Y in the source domain follows a probability distribution P S ( x , y ) .
Similarly, each target test sample and the corresponding label at test time t , ( x t , y t )
∈
X T × Y ,follows a probability distribution P T ( x , y ) where y t is unknown for the learner.
The standard covariateshift assumption in domain adaptation is deﬁned as P S ( x ) (cid:54) =
P T ( x ) and P S ( y | x ) =
P T ( y | x ) [26].
Unlike traditional domain adaptation that uses D S and X T collected beforehand for adaptation,test-time adaptation (TTA) continually adapts a pre-trained model f θ ( · ) from D S by utilizing only x tobtained at test time t .
TTA on non-i.i.d.
streams.
Note that previous TTA mechanisms typically assume that each targetsample ( x t , y t )
∈
X T ×Y is independent and identically distributed (i.i.d.)
following a time-invariantdistribution P T ( x , y ) .
However, the data obtained at test time is non-i.i.d.
in many scenarios.
Bynon-i.i.d., we refer to distribution changes over time, i.e., ( x t , y t ) ∼ P T ( x , y | t ) , which is a practicalsetting in many real world applications [40].
2.2 Batch normalization in TTA
Batch Normalization (BN) [14] is a widely-used training technique in deep neural networks as itreduces the internal covariant shift problem.
Let f ∈ R B × C × L denote a batch of feature maps ingeneral, where B , C , and L denote the batch size, the number of channels, and the size of eachfeature map, respectively.
Given the statistics of the feature maps for normalization, say mean µ andvariance σ 2 , BN is channel-wise , i.e., µ , σ 2 ∈ R C and computes:
[Equation]
where γ and β are the afﬁne parameters followed by the normalization, and (cid:15) > 0 is a small constantfor numerical stability.
Although a conventional way of computing BN in test-time is to set µ and σ 2 as those estimatedfrom training (or source) data, say ¯ µ and ¯ σ 2 , the state-of-the-art TTA methods based on adapting
BN layers [23, 27, 35, 38] instead use the statistics computed directly from the recent test batch tode-bias distributional shifts at test-time, i.e.:
[Equation]
This practice is simple yet effective under distributional shifts and is thus adopted in many recent
TTA studies [23, 27, 35, 38].
Based on the test batch statistics, they often further adapt the afﬁneparameters via entropy minimization of the model outputs [35] or update the entire parameters withself-training [38].
Method
In the same vein as previous work [23, 27, 35], we focus on adapting BN layers in the given model toperform TTA, and this includes essentially two approaches: (a) re-calibrating (or adapting)
channelwise statistics for normalization (instead of using those learned from training), and (b) adapting theafﬁne parameters (namely, γ and β ) after the normalization with respect to a certain objective basedon test samples, e.g., the entropy minimization of model outputs [35].
Under scenarios where test data are temporally correlated, however, naïvely adapting to the incomingbatch of test samples [23, 27, 35, 38] could be problematic for both approaches: the batch is now morelikely to (a) remove instance-wise variations that are actually useful to predict y , i.e., the “contents”rather than “styles” through normalization, and (b) include a bias in p ( y ) rather than uniform, whichcan negatively affect the test-time adaptation objective such as entropy minimization.
Out of distribution
In distribution 𝐈𝐍 𝐈𝐀𝐁𝐍 𝐁𝐍 ഥ𝝁 c ,ഥ𝝈 c ෥𝝁 b,c ,෥𝝈 b,c 𝐁𝐍 𝐈𝐀𝐁𝐍 𝐈𝐍 ෥𝝁 b,c ,෥𝝈 b,c ഥ𝝁 c ,ഥ𝝈 c
Non-i.i.d.
(temporally correlated) 𝒙 𝑡 𝑡𝑖𝑚𝑒 … … (1) Inference via 𝐈𝐀𝐁𝐍 ෝ𝝁 𝑡 ,ෝ𝝈 𝑡 (3) Adaptation via 𝐏𝐁𝐑𝐒 𝝁 𝑡 ← 𝝁 𝑡−1 ,ෝ𝝁 𝑡 𝝈 𝑡 ← 𝝈 𝑡−1 ,ෝ𝝈 𝑡 (𝝁 𝑡 ,𝝈 𝒕 ) (𝝁 𝑡−1 ,𝝈 𝑡−1 )
Time-uniform
P r ed i c t i on - un i f
o r m
Old
New
Class 𝒃
Class 𝒂 (2) Manage memory by 𝐏𝐁𝐑𝐒 (𝒙 𝑖 , ො𝑦 𝑖 )
Class 𝒄 (𝒙 𝑡 , ො𝑦 𝑡 )
We propose two approaches to tackle each of the failure modes of adapting BN under temporalcorrelation.
Our methods consists of two components: (a) Instance-Aware Batch Normalization(IABN) (Section §3.1) to overcome the limitation of BN under distribution shift and (b)
PredictionBalanced Reservoir Sampling (PBRS) (Section §3.2) to combat with the temporal correlation of testbatches. Figure 3 illustrates the overall workﬂow of NOTE with IABN and PBRS.
3.1 Instance-Aware Batch Normalization
As described in Section §2.2, recent TTA algorithms rely solely on the test batch to re-calculate BNstatistics.
We argue that this common practice does not successfully capture the feature statistics tonormalize the feature map f ∈ R B × C × L under temporal correlation in the test batch B .
In principle,standardizing a given feature map f : ,c, : by the statistics ˆ µ c , ˆ σ 2 c computed across B and L is positedon premise that averaging information across B can marginalize out uninformative instance-wisevariations for predicting y .
Under temporal correlation in B , however, this assumption is no longervalid, and averaging across B may not successfully de-correlate useful information in f : ,c, : from µ cand σ 2 c .
In an attempt to bypass such an “over-whitening” effect of using ˆ µ c and ˆ σ 2 c in test-time under temporalcorrelation, we propose correcting normalization statistics on a per-sample basis : speciﬁcally, insteadof completely switching from the original statistics of (¯ µ , ¯ σ 2 ) into (ˆ µ c , ˆ σ 2 c ) , our proposed InstanceAware Batch Normalization (IABN) considers the instance-wise statistics ˜ µ , ˜ σ 2 ∈ R B,C of f similarlyto Instance Normalization (IN) [31], namely:
[Equation]
We assume that ˜ µ b,c and ˜ σ 2 b,c follow the sampling distribution of a sample size L in N (¯ µ , ¯ σ 2 ) as thepopulation.
Then the corresponding variances for the sample mean ˜ µ b,c and the sample variance ˜ σ 2 b,ccan be calculated as:
[Equation]
IABN corrects (¯ µ , ¯ σ 2 ) only in cases when ˜ µ b,c
(and ˜ σ 2 b,c ) signiﬁcantly differ from ¯ µ c (and ¯ σ 2 c ).
Speciﬁcally, we propose to use the following statistics for TTA:
[Equation]
[Equation]
where ψ ( x ; λ ) =   x − λ, if x > λ x + λ, if x < − λ {+0 , otherwise is the soft-shrinkage function and k ≥ 
+}
is a hyperparameter.
Observe that IABN becomes IN and BN when k = 0 and k = ∞ , respectively.
If one chooses too
Input: target stream x t ∼ P T ( x | t ) , 1: for t ∈ { 1 ...N } do 2: if memory is not full then 3:
Add ( x t , ˆ y t ) 4: else 5:
L ← majority class(es) in memory 6: if ˆ y t / ∈ L then (cid:46) Prediction-Balanced 7:
Randomly discard one instance ( x i , ˆ y i ) in memory where ˆ y i ∈
L 8: Add ( x t , ˆ y t ) 9: else (cid:46) Reservoir Sampling 10: m c ← number of instances in memory where ˆ y i
= ˆ y t 11: n c ← number of instances encountered so far where ˆ y i
= ˆ y t 12:
Sample p ∼ Uniform(0,1) 13: if p < m c /n
c then 14:
Discard a random instance ( x i , ˆ y i ) in memory where ˆ y i
= ˆ y t 15:
Add ( x t , ˆ y t ) 16: else 17:
Discard ( x t , ˆ y t ) small k ≥ 0 , IABN may remove useful features, e.g., styles, of input (as with IN), which candegrade the overall classiﬁcation (or regression) performance [24].
Hence, it is important to choosean appropriate k .
Nevertheless, we found that a good choice of k is not too sensitive across testedscenarios, where we chose k = 4 for all experiments.
This way, IABN can be robust to distributionalshifts without the risk of eliminating crucial information to predict y .
3.2 Adaptation via Prediction-Balanced Reservoir Sampling
Temporally correlated distributions lead to an undesirable bias in p ( y ) , and thus adaptation witha batch of consecutive test samples negatively impacts the adaptation objective, such as entropyminimization [35].
To combat this imbalance, we propose Prediction-Balanced Reservoir Sampling(PBRS) that mimics i.i.d.
samples from temporally correlated streams with the assistance of a small(e.g., a mini-batch size) memory.
PBRS combines time-uniform sampling and prediction-uniformsampling to simulate i.i.d.
samples from the non-i.i.d.
streams.
For time-uniform sampling, we adoptreservoir sampling (RS) [34], a proven random sampling algorithm to collect time-uniform data in asingle pass on a stream without the prior knowledge of the total length of data.
For prediction-uniformsampling, we ﬁrst use the predicted labels to compute the majority class(es) in the memory.
We thenreplace a random instance of the majority class(es) with a new sample.
We detail the algorithm of
PBRS as a pseudo-code in Algorithm 1.
We found that these two heuristics can effectively balancesamples among both time and class axes, which mitigates the bias in temporally correlated data.
With the stored samples in the memory, we update the normalization statistics and afﬁne parameters inthe IABN layers.
Note that IABN assumes ˜ µ b,c and ˜ σ 2 b,c follow the sampling distribution of N (¯ µ , ¯ σ 2 )and corrects the normalization if ˜ µ b,c and ˜ σ 2 b,c are out of distribution.
While IABN is resilient todistributional shifts to a certain extent, the assumption might not hold under severe distributionalshifts.
Therefore, we aim to ﬁnd better estimates of ¯ µ , ¯ σ 2 in IABN under distributional shifts via
PBRS. Speciﬁcally, we update the normalization statistics, namely the means µ and variances σ 2 , viaexponential moving average: (a) µ t =
(1 − α ) µ t − 1 + α n t n t − 1 ˆ µ t and (b) σ 2 t =
(1 − α ) σ 2 t − 1 + α n t n t − 1 ˆ σ 2 twhere α is a momentum and n t is the size of the memory.
We further optimize the afﬁne parameters,scaling factor γ and bias term β via entropy minimization, similar to a previous study [35].
Experiments
We implemented NOTE and the baselines via the PyTorch framework [25].
We ran all experimentswith three random seeds and speciﬁed the standard deviations.
Additional experimental details are inthe material.
[Table] Baselines.
We consider the following baselines including the state-of-the-art test-time adaptationalgorithms: Source evaluates the model trained from the source data directly on the target data withoutadaptation.
Test-time normalization ( BN stats ) [23, 27] updates the BN statistics from a batch of testdata.
Online Domain Adaptation ( ONDA ) [22] adapts batch normalization statistics to target domainsvia a batch of target data with exponential moving average.
Pseudo-Label ( PL ) [18] optimizes thetrainable parameters in BN layers via hard pseudo labels.
We update the BN layers only in PL, asdone in previous studies [35, 38].
Test entropy minimization ( TENT )
[35] updates the BN parametersvia entropy minimization.
Laplacian Adjusted Maximum-likelihood Estimation ( LAME ) [4] takesa more conservative approach; it modiﬁes the classiﬁer’s output probability and not the internalparameters of the model itself.
By doing so, it prevents the model parameters from over-adaptingto the test batch.
Continual test-time adaptation ( CoTTA ) [38] reduces the error accumulation byusing weight-averaged and augmentation-averaged predictions.
It avoids catastrophic forgetting bystochastically restoring a part of the neurons to the source pre-trained weights.
Adaptation and hyperparameters.
We assume the model pre-trained with source data is availablefor TTA.
In NOTE, we replaced BN with IABN during training.
We set the test batch size as 64and epoch as one for adaptation, which is the most common setting among the baselines [27, 4, 35].
Similarly, we set the memory size as 64 and adapt the model every 64 samples in NOTE to ensure afair memory constraint.
We conduct online adaptation and evaluation, where the model is continuallyupdated.
For the baselines, we adopt the best values for the hyperparameters reported in their papersor the ofﬁcial codes.
We followed the guideline to tune the hyperparameters when such guidelinewas available [38].
We use ﬁxed values for the hyperparameters for IABN layers, soft-shrinkagewidth k = 4 and momentum α = 0 .
01 , and update the afﬁne parameters via the Adam optimizer [16]with a learning rate l = 0 . for all experiments. We detailed hyperparameter information of thebaselines in the supplementary material.
4.1 Robustness under corruptions
Datasets.
CIFAR10-C/CIFAR100-C [11] are common TTA benchmarks for evaluating the robustness to corruptions [23, 27, 35, 38].
Both CIFAR10/CIFAR100 [17] have 50,000/10,000 training/testdata.
CIFAR10/CIFAR100 have 10/100 classes respectively.
CIFAR10-C/CIFAR100-C apply 15types of corruptions to CIFAR10/CIFAR100 test data.
Similar to previous studies [23, 27, 35, 38],we use the most severe corruption level of 5.
We use ResNet18 [10] as the backbone network andpre-trained it on the clean training data.
Following prior studies [19, 13, 37, 36], we adopt Dirichletdistribution to generate synthetic non-i.i.d.
test streams from the originally i.i.d.
CIFAR10/100 data.
The details are provided in the supplementary material.
We vary the Dirichlet concentration parameterδ to simulate diverse streams and visualize the resulting data in Figure 4.
We use δ = 0 .
1 as thedefault value unless speciﬁed.
Overall result.
Tables 1 shows the result under the temporally correlated (non-i.i.d.)
data and theuniform (i.i.d.) data, respectively.
We observe signiﬁcant performance degradation in the baselines
[Figure] D i r i c h l e t pa r a m e t e r 𝛿 0. [Figure]
[Figure]
    [Figure] 5QWTEG$05VCVU2.10&# 6'06.#/'%Q66#
[Figure]
under the temporally correlated setting.
For BN Stats, PL, TENT, and CoTTA, this degradation isparticularly due to the dependence on the test batch for the re-calculation of the BN statistics.
Updatingthe batch statistics from test data via exponential moving average (ONDA) also suffers from thetemporally correlated data.
This indicates relying on the test batch for re-calculating the BN statisticsindeed cancels out meaningful instance-wise variations under temporal correlation.
Interestingly,
LAME works better in the non-i.i.d. setting than in the i.i.d.
setting, which is consistent with thereports from the paper [4].
NOTE achieves on average 15.7% improvement over the best baseline(LAME) under the non-i.i.d.
setting.
With the i.i.d.
assumption, NOTE still achieves comparableperformance to the baselines.
When we know the target samples are i.i.d.
, we can simply use the testbatch without using PBRS.
We report the result of the variant version of NOTE in the parentheses,which achieves on average 3.9% improvement further when the i.i.d.
assumption is known.
Effect of the degree of temporal correlation.
We also investigate the effect of the degree oftemporal correlation for TTA algorithms.
Figure 5a shows the result.
The lower δ is, the severertemporal correlation becomes.
The error rates of most of the baselines deteriorate as δ decreases,which shows that the existing TTA baselines are susceptible to temporally correlated data.
NOTEshows consistent performance among all δ values, indicating its robustness under temporal correlation.
Effect of batch size.
While we experiment with a widely-used value of 64 as the batch size (ormemory size in NOTE), one might be curious about the impact of batch size under temporallycorrelated streams.
Figure 5b shows the result with six different batch sizes.
As shown, NOTE is notmuch affected by the batch size, while most of the baselines recovers performance degradation asthe batch size increases.
This is because a higher batch size has a better chance of adaptation withbalanced samples under temporally correlated streams.
Increasing the batch size, however, mitigatestemporal correlation at the expense of inference latency and adaptation speed.
4.2 Real distributions with domain shift
Datasets.
We evaluate NOTE under three real-world distribution datasets: object detection inautonomous driving (KITTI [8]), human activity recognition (HARTH [21]), and user behavioralcontext recognition (ExtraSensory [32]).
KITTI is a well-known autonomous driving dataset thatprovides consecutive frames that contains natural temporal correlation in driving contexts.
We adaptfrom KITTI to KITTI-Rain [9] - a dataset that converted KITTI images to rainy images.
This contains7,481/7,800 train/test samples with nine classes.
We use ResNet50 [10] pre-trained on ImageNet [7]as the backbone network.
HARTH was collected from 22 users under free-living environments forseven days.
Each user was equipped with two three-axial Axivity AX3 accelerometers for recordinghuman activities.
We use 15 users collectively as the source domain and the remaining seven users aseach target domain, which entails natural domain shift from source users to target users as differentphysical conditions make domain shifts across users.
HARTH contains 82,544/39,377 train/testsamples with 12 classes.
We report the averaged error over all target domains.
We use four one-
[Table]
[Table]
[Table] dimensional convolutional layers followed by one fully-connected layer as the backbone networkfor HARTH. Extrasensory dataset collected users’ own smartphone sensory data (motion sensors,audio, etc.) in the wild for seven days, aiming to capture people’s authentic behaviors in their regularactivities.
We use 16 users as the source domain and seven users as the target domains.
ExtraSensoryincludes 17,777/4,862 train/test data with ﬁve classes.
For ExtraSensory, we use two one-dimensionalconvolutional layers followed by one fully-connected layer as the backbone network.
Additionaldetails are in the supplementary material.
Result.
Table 2 shows the result for the real-world datasets.
The overall trend is similar to thetemporal correlation experiments with CIFAR10-C/CIFAR100-C datasets, which indicates thatthe real-world datasets are indeed temporally correlated.
NOTE consistently reduces errors afteradaptation under real-world distributions.
We believe this demonstrates NOTE is a promising methodto be utilized in various real-world ML applications with distributional shifts.
4.3 Ablation study
We conduct an ablative study to further investigate the effectiveness of the individual components.
Table 3 shows the result under both i.i.d.
and non-i.i.d.
settings.
Using IABN alone signiﬁcantlyreduces error rates over Source, demonstrating the effectiveness of correcting normalization for outof-distribution samples.
Using PBRS with BN shows comparable improvement with the IABN-onlyresult.
Note that there is only a marginal gap (around 1%) between the non-i.i.d.
and i.i.d.
resultsin PBRS.
This indicates that PBRS could effectively simulate i.i.d.
samples from non-i.i.d.
streams.
The joint use of IABN and PBRS outperforms using either of them, meaning that PBRS provides
IABN with better estimates for the normalizing operation.
In addition, PBRS is better than Reservoir
Sampling (RS) that has been a strong baseline in continual learning [15, 5].
This shows storingprediction-balanced sampling in addition to time-uniform sampling leads to better adaptation in
TTA.
We also investigated the joint use of IN and PBRS with the combination of IABN and PBRSon CIFAR100-C, and the result shows that IABN+PBRS (47.0%) achieves lower error rate than
IN+PBRS (52.5%) on CIFAR100-C under temporal correlation.
Related work
Test-time adaptation.
Test-time adaptation (TTA) attempts to overcome distributional shifts withtest data without the cost of data acquisition or labeling.
TTA adapts to the target domain with only testdata on the ﬂy.
Most existing TTA algorithms rely on a batch of test samples to adapt [23, 27, 35, 38]to re-calibrate BN layers on the test data.
Simply using the statistics of test batch in BN layersimproves the robustness under distributional shifts [23, 27].
ONDA [22] updates the BN statisticswith test data via exponential moving average. TENT [35] further updates the scaling and biasparameters in BN layers via entropy minimization. Latest TTA studies consider distribution changesof test data [4, 38]. LAME [4] corrects the output probabilities of a classiﬁer rather than tweakingthe model’s inner parameters.
By restraining the model from over-adapting to the test batch, LAMEallows the model to be more robust under non-i.i.d. scenarios.
However, LAME does not havenoticeable performance gains in class-balanced, standard i.i.d.
scenarios.
CoTTA [38] aims to adaptto continually changing target environments via weight-averaged teacher model, weight-averagedaugmentations, and stochastic restoring.
However, CoTTA assumes i.i.d.
test data within each domainand updates the entire model that increases computational costs.
There also exist works [29, 20]utilizing domain-speciﬁc self-supervision to resolve the distribution shift with test data, but arecomplementary to ours, i.e., we can also optimize the self-supervised loss instead of the entropy loss,and not applicable to our setups of real test data stream as designing good self-supervision for thesedomains is highly non-trivial.
Replay memory.
Replay memory is one of the major approaches in continual learning; it managesa buffer to replay previous data for future learning to prevent catastrophic forgetting.
Reservoirsampling [34] is a random sampling algorithm that collects time-uniform samples from unknown nsamples with a single pass, and it has been proven to be a strong baseline in continual learning [15, 5].
GSS [1] stores samples to a memory in a way that maximizes the gradient direction among thosesamples.
A recent study modiﬁes reservoir sampling to balance classes under imbalanced data, whenthe labels are given [6].
Our memory management scheme (PBRS) is inspired by these studies toprevent catastrophic forgetting in test-time adaptations.
Discussion and conclusion
This paper highlights that real-world distributions often changes across the time axis, and existing testtime adaptation algorithms mostly suffer from the non-i.i.d.
test data streams.
To address this problem,we present a NOn-i.i.d.
TEst-time adaptation algorithm, NOTE.
Our experiments evaluated robustnessunder corruptions and domain adaptation on real-world distributions.
The results demonstrate that
NOTE not only outperforms the baselines under the non-i.i.d./real distribution settings, but it alsoshows comparable performance (even surpassing in some settings) under the i.i.d.
assumption.
Webelieve that the insights and ﬁndings from this study is a meaningful step towards the practical impactof test-time adaptation paradigm.
Limitations.
Similar to existing TTA algorithms [23, 27, 35, 38], we assume that the backbonenetworks are equipped with BN layers, and particularly, we replaced the BN layers with IABN layers.
While BN is a widely-used component in deep learning, there exist architectures that do not embed
BN layers such as LSTMs [12] and Transformers [33]. Whether naively inserting BN or IABN wouldbe sufﬁcient for applying these TTA methods is still in question.
A recent study evidenced that BN isadvantageous in Vision Transformers [39], showing potential room to apply our idea to architectureswithout BN layers.
However, more in-depth studies are necessary to identify the actual applicabilityof BN (or IABN) to those architectures.
Potential negative societal impacts.
As TTA relies on unlabeled test samples and changes themodel accordingly, the model is exposed to potential data-driven biases after adaptation, such asfairness issues [3] and adversarial attacks [30].
In some sense, the utility of TTA comes at theexpense of exposure to threats.
This vulnerability is another crucial problem that both ML researchersand practitioners need to take into consideration.
In addition, TTA entails additional computationsfor adaptation with test data, which may have negative impacts on environments, e.g., increasingelectricity consumption and carbon emissions [28].
Nevertheless, we believe NOTE would notexacerbate this issue as it is computationally efﬁcient as mentioned in Section §1.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope? [Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section §6.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] SeeSection §6.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them? [Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex-
perimental results (either in the supplemental material or as a URL)?
[Yes] See the
supplementary material.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they were chosen)? [Yes] See Section §4 and the supplementary material.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] We ran the entire experiments with three different random
seems (0,1,2) and reported the standard deviations.
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)? [Yes] See the supplementary material.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[Yes]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[Yes]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
you describe any potential participant risks, with links to Institutional Review
Board
approvals, if applicable /A]
Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]