Title: SoTTA: Robust Test-Time Adaptation on Noisy Data Streams
Venue: NeurIPS 2023
Accepted: yes
Keywords: test-time adaptation

domain adaptation

deep learning

machine learning
Abstract: Test-time adaptation (TTA) aims to address distributional shifts between training and testing data using only unlabeled test data streams for continual model adaptation. However, most TTA methods assume benign test streams, while test samples could be unexpectedly diverse in the wild. For instance, an unseen object or noise could appear in autonomous driving. This leads to a new threat to existing TTA algorithms; we found that prior TTA algorithms suffer from those noisy test samples as they blindly adapt to incoming samples. To address this problem, we present Screening-out Test-Time Adaptation (SoTTA), a novel TTA algorithm that is robust to noisy samples. The key enabler of SoTTA is two-fold: (i) input-wise robustness via high-confidence uniform-class sampling that effectively filters out the impact of noisy samples and (ii) parameter-wise robustness via entropy-sharpness minimization that improves the robustness of model parameters against large gradients from noisy samples. Our evaluation with standard TTA benchmarks with various noisy scenarios shows that our method outperforms state-of-the-art TTA methods under the presence of noisy samples and achieves comparable accuracy to those methods without noisy samples. The source code is available at https://github.com/taeckyung/SoTTA.

SoTTA: Robust Test-Time Adaptation on Noisy Data Streams

Abstract

Test-time adaptation (TTA) aims to address distributional shifts between training and testing data using only unlabeled test data streams for continual model adaptation. However, most TTA methods assume benign test streams, while test samples could be unexpectedly diverse in the wild. For instance, an unseen object or noise could appear in autonomous driving. This leads to a new threat to existing TTA algorithms; we found that prior TTA algorithms suffer from those noisy test samples as they blindly adapt to incoming samples. To address this problem, we present Screening-out Test-Time Adaptation (SoTTA), a novel TTA algorithm that is robust to noisy samples. The key enabler of SoTTA is two-fold: (i) inputwise robustness via high-confidence uniform-class sampling that effectively filters out the impact of noisy samples and (ii) parameter-wise robustness via entropysharpness minimization that improves the robustness of model parameters against large gradients from noisy samples. Our evaluation with standard TTA benchmarks with various noisy scenarios shows that our method outperforms state-of-the-art TTA methods under the presence of noisy samples and achieves comparable accuracy to those methods without noisy samples. The source code is available at https://github.com/taeckyung/SoTTA .

1 Introduction

Deep learning has achieved remarkable performance in various domains [6, 8, 33], but its effectiveness is often limited when the test and training data distributions are misaligned. This phenomenon, known as domain shift [31], is prevalent in real-world scenarios where unexpected environmental changes and noises result in poor model performance. For instance, in autonomous driving, weather conditions can change rapidly. To address this challenge, Test-Time Adaptation (TTA) [1, 5, 29, 38, 39, 44] has emerged as a promising paradigm that aims to improve the generalization ability of deep learning models by adapting them to test samples, without requiring further data collection or labeling costs.

While TTA has been acknowledged as a promising method for enhancing the robustness of machine learning models against domain shifts, the evaluation of TTA frequently relies on the assumption that the test stream contains only benign test samples of interest. However, test data can be unexpectedly diverse in real-world settings, containing not only relevant data but also extraneous elements that are outside the model's scope, which we refer to noisy samples (Figure 1). For instance, unexpected noises can be introduced in autonomous driving scenarios, such as dust on the camera or adversarial samples by malicious users. As shown in Figure 2, we found that most of the prior TTA algorithms showed significantly degraded accuracy with the presence of noisy samples (e.g., 81.0% → 52.1% in TENT [38] and 82.2% → 54.8% in CoTTA [39]).

Benign sample

N

oisy

sample

t

Online data stream

Acc: 81%

Clean TTA

Noisy TTA

Update

Update

Acc: 52%



















SoTTA











To ensure the robustness of TTA against noisy samples, an intuitive solution might be screening out noisy samples from the test stream. Out-of-distribution (OOD) detection [10, 11, 18, 19, 20, 21, 24, 25, 43] is a representative method for this, as it tries to detect whether a sample is drawn from the same distribution as the training data or not. Similarly, open-set domain adaptation (OSDA) [30, 35] and universal domain adaptation (UDA) [34, 42] generalize the adaptation scenario by assuming that unknown classes are present in test data that are not in training data. However, these methods require access to a whole batch of training data and unlabeled target data, which do not often comply with TTA settings where the model has no access to train data at test time due to privacy issues [38] and storing a large batch of data is often infeasible due to resource constraints [12]. Therefore, how to make online TTA robust under practical noisy settings is still an open question.

In this paper, we propose Screening-out Test-Time Adaptation (SoTTA) that is robust to noisy samples. SoTTA achieves robustness to noisy samples in two perspectives: (i) input-wise robustness and (ii) parameter-wise robustness. Input-wise robustness aims to filter out noisy samples so that the model will be trained only with benign samples. We achieve this goal via High-confidence Uniformclass Sampling (HUS) that avoids selecting noisy samples when updating the model (Section 3.1). Parameter-wise robustness pursues updating the model weights in a way that prevents model drifting due to large gradients caused by noisy samples. We achieve this via entropy-sharpness minimization (ESM) that makes the loss landscape smoother and parameters resilient to weight perturbation caused by noisy samples (Section 3.2).

We evaluate SoTTA with three common TTA benchmarks (CIFAR10-C, CIFAR100-C, and ImageNetC [9]) under four noisy scenarios with different levels of distributional shifts: Near, Far, Attack, and Noise (Section 2). We compare SoTTA with eight state-of-the-art TTA algorithms [1, 17, 27, 28, 29, 38, 39, 44], including the latest studies that address temporal distribution changes in TTA [1, 28, 29, 39, 44]. SoTTA showed its effectiveness with the presence of noisy samples. For instance, in CIFAR10-C, SoTTA achieved 80.0% accuracy under the strongest shift case (Noise), which is a 22.3%p improvement via TTA and 6.4%p better than the best baseline [44]. In addition, SoTTA achieves comparable performance to state-of-the-art TTA algorithms without noisy samples, e.g., showing 82.2% accuracy when the best baseline's accuracy is 82.4% in CIFAR10-C.

Contributions. (i) We highlight that test sample diversity in real-world scenarios is an important problem but has not been investigated yet in the literature. We found that most existing TTA algorithms undergo significant performance degradation with sample diversity. (ii) As a solution, we propose SoTTA that is robust to noisy samples by achieving input-wise and parameter-wise robustness. (iii) Our evaluation with three TTA benchmarks (CIFAR10-C, CIFAR100-C, and ImageNet-C) show that SoTTA outperforms the existing baselines.

(a) Benign.

(b) Near.

(c) Far.

(d) Attack.

司

2

(a) Benign sample

(b)

Noisy s

ample

cat dog bird

𝜃

!

𝜃

!

𝜃

!"#

$%&

𝜃

!"#

𝜃

!

Online data stream

cat dog bird

(a) Input-wise robustness

Update

𝜃

!

(b) Parameter-wise robustness

Cat

Dog

Bird

αo

αo

2 Preliminaries

Test-time adaptation. Let D S = {X S , Y} be source data and ( x i , y i ) ∈ X S ×Y be each instance and the label pair that follows a probability distribution of the source data P S ( x , y ) . Similarly, let D T = {X T , Y} be target data and ( x j , y j ) ∈ X T ×Y be each target instance and the label pair following a target probability distribution P T ( x , y ) , where y j is usually unknown to the learning algorithm. The covariate shift assumption [31] is given between source and target data distributions, which is defined as P S ( x ) = P T ( x ) and P S ( y | x ) = P T ( y | x ) . Given an off-the-shelf model f ( · ; Θ) pre-trained from D S , the (fully) test-time adaptation (TTA) [38] aims to adapt f ( · ; Θ) to the target distribution P T utilizing only x j given test time.

̸

Noisy test samples. We define noisy test samples to represent any samples that are not included in the target data distribution, i.e., ˜ x / ∈ X T . We use the term noisy to distinguish it from out-of-distribution (OOD), as TTA typically aims to adapt to OOD samples, such as corrupted ones. Theoretically, there could be numerous categories of noisy samples. In this study, we consider five scenarios: Benign, Near, Far 2 , Attack, and Noise. Figure 3 shows examples of these scenarios. Benign is the typical setting of TTA studies without noisy samples, Near represents a semantic shift [41] from target distribution, Far is a severer shift where covariate shift is evident [41], Attack refers to intelligently generated adversarial attack with perturbation [40], and Noise refers to random noise. We focus on these scenarios to understand the impact of noisy samples in TTA as well as the simplicity of analysis. Detailed settings are described in Section 4.

3 Methodology

Problem and challenges. Prior TTA methods assume test samples are benign and blindly adapt to incoming batches of test samples. The presence of noisy samples during test time can significantly degrade their performance for those TTA algorithms, which has not been explored in the literature yet. Dealing with noisy samples in TTA scenarios is particularly challenging as (i) TTA has no access to the source data, (ii) no labels are given for target test data, and (iii) the model is continually adapted and thus a desirable solution should apply to varying models. This makes it difficult to apply existing solutions that deal with a similar problem. For instance, out-of-distribution (OOD) detection studies [11, 19, 20, 43] are built on the assumption that a model is fixed at test time, and open-set domain adaptation (OSDA) methods [30, 35, 42] require labeled source and unlabeled target data for training.

Methodology overview. To address the problem, we propose Screening-out Test-Time Adaptation (SoTTA), whose overview is described in Figure 4. SoTTA achieves robustness to noisy samples in two perspectives: (i) input-wise robustness via high-confidence uniform-class sampling that avoids selecting noisy samples when updating the model (Section 3.1), and (ii) parameter-wise robustness via entropy-sharpness minimization that makes parameters resilient to weight perturbation caused by noisy samples (Section 3.2).

3.1 Input-wise robustness via high-confidence uniform-class sampling

Our first approach is to ensure inputwise robustness to noisy samples by filtering out them when selecting samples for adaptation. As locating noisy samples without their labels is challenging, our idea is based on the empirical observation of the model predictions with respect to noisy samples.

Observation. Our hypothesis is that noisy samples have distinguished properties from benign samples due to the distributional shifts, and this could

0.2

0.4

0.6

0.8

1.0

0.0

5.0

10.0

15.0

20.0

Density

Benign

Near

Far

Attack

Noise

(a) Confidence distribution.

0 1 2 3 4 5 6 7 8 9

Class

Proportion (%)

Benign

Near

Far

Attack

Noise

0

20

40

60

80

(b) Predicted class distribution.

be observable via the models' prediction outputs. We investigate two types of features that work as proxies for identifying benign samples: (i) confidence of the samples and (ii) predicted class distributions. Specifically, we compared the distribution of the softmax confidence (Figure 5a) and the predicted class distribution (Figure 5b) of benign samples with noisy samples. First, the confidence of the samples is relatively lower than that of benign samples. The more severe the distribution shift, the lower the confidence (e.g., Far is less confidence than Near), which is also in line with findings of previous studies that pre-trained models show higher confidence on target distribution than out-of-distribution data [10, 21]. Second, we found that noisy samples are often skewed in terms of predictions, and this phenomenon is prominent in more severe shifts (e.g., Noise), except for Attack, whose objective is to make the model fail to correctly classify. These skewed distributions could lead to an undesirable bias in p ( y ) and thus might negatively impact the TTA objective, such as entropy minimization [38].

Algorithm 1 High-confidence Uniform-class Sampling (HUS)

Solution. Based on the aforementioned empirical analysis, we propose High-confidence Uniformclass Sampling (HUS) that avoids using noisy samples for adaptation by utilizing a small memory. We maintain confident samples while balancing their predicted classes in the memory. The selected samples in the memory are then used for adaptation. We describe the procedure as a pseudo-code code in Algorithm 1. Given a target test sample x , HUS measures its confidence. Specifically, we define the confidence C ( x ; Θ) of each test sample x as:

We store the sample if its confidence is higher than the predefined threshold C 0 . In this way, we maintain only high-confidence samples in the memory used for adaptation and thus reduce the impact of low-confidence noisy samples.

Furthermore, while storing data in the memory, we balance classes among them. Specifically, if the predicted class of the current test sample is not in the most prevalent class in the memory, then HUS randomly replaces one random sample in the most prevalent class with the new sample. Otherwise, if the current sample belongs to the most prevalent class in the memory, HUS replaces one random sample in the same class with the current one. We can maintain classes uniformly with this strategy, which is effective for not only filtering out noisy samples but also removing class biases among samples when used for adaptation, which we found is beneficial for TTA. We found these two memory management strategies not only effectively reduce the impact of noisy samples for adaptation but also improve the model performance in benign-only cases by avoiding model drifting due to biased and low-confidence samples (Section 4).

With the stored samples in the memory, we update the normalization statistics and affine parameters in batch normalization (BN) [13] layers, following the prior TTA methods [29, 38, 44]. This is known as not only computationally efficient but also showed comparable performance improvement to updating the whole layers. While we aim to avoid using noisy samples for adaptation, a few noisy samples could still be stored in the memory, e.g., when they are similar to benign samples or outliers. To be robust to temporal variances of the samples in the memory, we take the exponential moving average to update the BN statistics (means µ and variances σ 2 ) instead of directly using the statistics from the samples in the memory. Specifically, we update the means and variances of BN layers as: (i) ˆ µ t = (1 -m )ˆ µ t -1 + mµ t and (ii) ˆ σ 2 t = (1 -m )ˆ σ 2 t -1 + mσ 2 t , where m ∈ [0 , 1] is a momentum hyperparameter. We describe updating the affine parameters in Section 3.2.

3.2 Parameter-wise robustness via entropy-sharpness minimization

Our second approach is to secure parameter-wise robustness to noisy samples by training the model in a way that is robust to noisy samples. Our idea is based on the observation that the parameter update is often corrupted with noisy samples, and this could be mitigated by smoothing the loss landscape with respect to parameters.

Observation. While most existing TTA algorithms utilize the entropy minimization loss [5, 38, 44], it could drift the model with high gradient

















































(a) Without ESM.

















































(b) With ESM.

samples [29]. We observed that adaptation with noisy samples often hinders the model from adapting to benign samples. Figure 6 shows an example of this phenomenon. Specifically, test samples consist of 10k benign samples and 10k noisy samples (Noise), which are randomly shuffled. We computed the cumulative accuracy for the benign test data and the gradient norm of noisy samples at each step. As shown in Figure 6a, the gradient norm of noisy samples dropped rapidly, indicating that the model is gradually adapting to these noise data. This leads to a significant accuracy drop for benign samples. The key question here is how to prevent the model from overfitting to noisy samples. Figure 6b shows the result with entropy-sharpness minimization (ESM) that we explain in the following paragraph. With ESM, the gradient norm of noise data remains high, and the accuracy for benign samples improved after adaptation as intended.

Solution. To make the model parameters robust to adaptation with noisy samples, the entropy loss landscape should be smoother so that the model becomes resilient to unexpected model drift due to noisy samples. To that end, we jointly minimize the naive entropy loss and the sharpness of the entropy loss and thus make the loss landscape robust to model weight perturbations by large gradients from noisy samples. Specifically, we replace the naive entropy minimization E with the

entropy-sharpness minimization (ESM) as:

where the entropy-sharpness E S ( x , Θ) is defined as the maximum objective around the weight perturbation with L2-norm constraint ρ . To tackle this joint optimization problem, we follow sharpnessaware minimization [4] similar to [29], which originally aims to improve the generalizability of models over standard optimization algorithms such as stochastic gradient descent (SGD). We repurpose this optimization to make the model robust to noisy samples in TTA scenarios.

Specifically, assuming ρ ≪ 1 , the optimization can be approximated via a first-order Taylor expansion:

The solution for this approximation problem is given by the classical dual norm problem:

where 1 /p + 1 /q = 1 . We set p = 2 for further implementation, following the suggestion [4]. By substituting ˆ ϵ (Θ) to the original entropy-sharpness minimization problem, the final gradient approximation is:

In summary, we calculate the entropy-sharpness minimization objective via two steps. First, at time step t , it calculates the ˆ ϵ (Θ t ) with previous parameters and entropy loss. It generates the temporal model with the new parameters: Θ t +ˆ ϵ (Θ t ) . Second, based on the temporary model, the second step updates the original model's parameters with the approximation in Equation 5. By putting it together with HUS (Section 3.1), parameters are updated by:

where η is the step size and t 0 is the model adaptation interval. For simplicity, we set the model adaptation interval the same as the memory capacity, i.e., t 0 = N . In the experiments, we use N = 64 , which is one of the most widely-used batch sizes in TTA methods [36, 38, 44].

4 Experiments

This section describes our experimental setup and demonstrates the results in various settings. Please refer to Appendix A and B for further details.

Scenario. To mimic the presence of noisy samples in addition to the original target test samples, we injected various noisy datasets into target datasets and randomly shuffled them, which we detail in the following paragraphs. We report the classification accuracy of the original target samples to measure the performance of the model in the presence of noisy samples. We ran all experiments with three random seeds (0, 1, 2) and reported the average accuracy and standard deviations in Appendix B.

Target datasets. We used three standard TTA benchmarks: CIFAR10-C , CIFAR100-C , and ImageNet-C [9] as our target datasets. All datasets contain 15 different types and five levels of corruption, where we use the most severe corruption level of five. For each corruption type, the CIFAR10-C/CIFAR100-C dataset has 10,000 test data with 10/100 classes, and the ImageNet-C dataset has 50,000 test data with 1000 classes. We use ResNet18 [8] as the backbone network. We pre-trained the model for CIFAR10 with training data and used the TorchVision [23] pre-trained model for ImageNet.

Noisy datasets. Besides the target datasets ( Benign ) mentioned above, we consider four noisy scenarios (Figure 3): CIFAR100 [15]/ImageNet [3] ( Near ), MNIST [26] ( Far ), adversarial attack ( Attack ), and uniform random noise ( Noise ). As both CIFAR10-C and ImageNet-C have real-world images for object recognition tasks, CIFAR100 would be a near dataset to them. For CIFAR100-C, we select ImageNet [3] for a near dataset. We select MNIST as a far dataset because they are not real-world images. We referred to the OOD benchmark [41] for choosing the term Near and Far and the datasets. For the adversarial attack, we adopt the Distribution Invading Attack (DIA), which is an

adversarial attack algorithm designed for TTA [40]. We inject the small perturbations to malicious samples to increase the overall error rate on benign data in the same batch. For the uniform random noise, we generate pixel-wise uniform-random images with the same size as the target images. We set the number of noisy samples equal to each target dataset as default, and we also investigated the impact of the number of noisy samples in the following ablation study (Figure 7). In cases where the number of noisy samples is different from the target datasets, we randomly resampled them. To ensure that the learning algorithm does not know the information of noisy samples beforehand, the pixel values of all noisy images are normalized with respect to the target dataset.

Baselines. We consider various state-of-the-art TTA methods as baselines. Source evaluates the pre-trained model directly on the target data without adaptation. Test-time batch normalization ( BN stats ) [27] updates the BN statistics from the test batch. Pseudo-Label ( PL ) [17] optimizes the trainable BN parameters via pseudo labels. Test entropy minimization ( TENT ) [38] updates the BN parameters via entropy minimization.

We also consider the latest TTA algorithms that improve robustness to temporal distribution changes in test streams to understand their performance to noisy samples. Laplacian adjusted maximumlikelihood estimation ( LAME ) [1] modifies the classifier output probability without modifying model internal parameters. Continual test-time adaptation ( CoTTA ) [39] uses weight-averaged and

augmentation-averaged predictions and avoids catastrophic forgetting by stochastically restoring a part of the model. Efficient anti-forgetting test-time adaptation ( EATA ) [28] uses entropy and diversity weight with Fisher regularization to prevent forgetting. Sharpness-aware and reliable optimization ( SAR ) [29] removes high-entropy samples and optimizes entropy with sharpness minimization [4]. Robust test-time adaptation ( RoTTA ) [44] utilizes the teacher-student model to stabilize while selecting the data with category-balanced sampling with timeliness and uncertainty.

Hyperparameters. We adopt the hyperparameters of the baselines from the original paper or official codes. We use the test batch size of 64 in all methods for a fair comparison. Accordingly, we set the memory size to 64 and adapted the model for every 64 samples for our method and RoTTA [44]. We conduct TTA in an online manner. We used a fixed hyperparameter of BN momentum m = 0 . 2 and updated the BN affine parameters via the Adam optimizer [14] with a fixed learning rate of l = 0 . 001 and a single adaptation epoch. The confidence threshold C 0 is set to 0.99 for CIFAR10-C, 0.66 for CIFAR100-C, and 0.33 for ImageNet-C. We set the sharpness threshold ρ = 0 . 05 as previous works [4, 29]. We specify further details of the hyperparameters in Appendix A.

Overall result. Table 1 shows the result on CIFAR10-C for 15 types of corruptions under five noisy scenarios described in Figure 3. We observed significant performance degradation in most of the baselines under the noisy settings. In addition, the extent of accuracy degradation from the Benign case is more prominent in more severe shift cases (e.g., the degree of degradation is generally Near < Far < Attack < Noise). Popular TTA baselines (BN stats, PL, and TENT) might fail due to updating with noisy samples. State-of-the-art TTA baselines that address temporal distribution shifts in TTA (LAME, CoTTA, EATA, SAR, and RoTTA) still suffered from noisy scenarios, as they are not designed to deal with unexpected samples. On the other hand, SoTTA showed its robustness across different scenarios. This validates the effectiveness of our approaches to ensure both input-wise and parameter-wise robustness. Interestingly, we found that SoTTA showed comparable performance to state-of-the-art baselines in the Benign case as well. Our interpretation of this result is two-fold. First, our high-confidence uniform-class sampling strategy filters not only noisy samples but also benign samples that would negatively impact the algorithm's objective. This implies that there exist samples that are more beneficial for adaptation, which aligns with the findings that high-entropy samples

harm adaptation performance [29]. Second, entropy-sharpness minimization helps ensure both the robustness to noisy samples and the generalizability of the model by preventing model drifts from large gradients, leading to performance improvement with benign samples. We found similar patterns for the CIFAR100-C (Table 2) and ImageNet-C (Table 3). More details are in Appendix B.

Impact of individual components of SoTTA. We conducted an ablative study to further investigate the effectiveness of SoTTA's individual components. Table 4 shows the result of the ablation study for CIFAR10-C. For input-wise robustness, HC refers to high-confidence sampling, and UC refers to uniform-class sampling. The two strategies are integrated into our high-confidence uniform-class sampling ( HUS ). ESM is our entropy-sharpness minimization for parameter-wise robustness. Note that we utilized FIFO memory with the same size for the UC case without HC and the native entropy minimization where we did not utilize ESM. Overall, the accuracy is improved as we sequentially added each approach of SoTTA. This validates our claim that ensuring both input-wise and parameterwise robustness via HUS and ESM is a synergetic strategy to combat noisy samples in TTA.

Impact of the number of noisy samples. We also investigate the effect of the size of noisy samples on TTA algorithms. Specifically, for the CIFAR10-C dataset with the Noise case, we varied the noise samples from 5,000 (5k) to 20,000 (20k) while fixing the size of benign samples as 10,000. Figure 7 shows the result. We observe that the accuracy of most baselines tends to deteriorate with a larger number of noisy samples and is sometimes even worse than without adaptation (Source). SoTTA showed its resilience to the size of noisy samples with 1.9%p degradation from 5k to 20k samples. RoTTA also showed its robustness to noisy samples to some extent, but the performance gain is around 6.4%p lower than SoTTA.

5 Related work

Test-time adaptation. While test-time adaptation (TTA) attempts to optimize model parameters with unlabeled test data streams, it lacks consideration of spoiling of the sample itself; i.e., it inevitably adapts to potential outlier samples mixed in the stream, such as adversarial data or mere noise. Most existing TTA algorithms directly optimize the model with incoming sample data. Test-time normalization [27, 36] updates the batch norm statistics in test-time to minimize the expected loss. TENT [38] minimizes the entropy of the model's predictions on a batch of test data.On the one hand, recent studies promote the robustness of the model, yet the consideration is limited to temporal distribution shifts of test data [1, 4, 5, 28, 29, 39, 44]. For instance, CoTTA [39] tries to adapt the model in a continually changing target environment via self-training and stochastical restoring. NOTE [5] proposes instance-aware batch normalization combined with prediction-balanced reservoir sampling to ensure model robustness towards temporally correlated data stream, which requires retraining with source data. We provide a method-wise comparison with EATA [28], SAR [29], and RoTTA [44] in Appendix D.2. To conclude, while existing TTA methods seek the robustness of the

















SoTTA





















model to temporal distribution shifts, they do not consider scenarios where noisy samples appear in test streams.

Out-of-distribution detection. Out-of-distribution (OOD) detection [10, 11, 18, 19, 20, 21, 24, 25, 43] aims to ensure the robustness of the model by identifying when a given data falls outside the training distribution. A representative method is a thresholding approach [10, 19, 20, 21], that defines a scoring function given an input and pretrained classifier. The sample is detected as OOD data if the output of the scoring function is higher than a threshold. Importantly, OOD detection studies are built on the condition that training and test domains are the same, which differs from TTA's scenario. Furthermore, OOD detection assumes that a model is fixed during test time, while a model changes continually in TTA. These collectively make it difficult to apply OOD detection studies directly to TTA scenarios.

Open-set domain adaptation. Open-set Domain Adaptation (OSDA) assumes that a target domain contains unknown classes not discovered in a source domain [30, 35] in domain adaptation scenarios. These methods aim to learn a mapping function between the source and target domains. While the target scenario that a model could encounter unknown classes of data in the test time is similar to our objective, these methods do not fit into TTA as it assumes both labeled source and unlabeled target data are available in the training time. Universal domain adaptation (UDA) [34, 42] further generalizes the assumption of OSDA by allowing unknown classes to present in both the source and the target domains. However, the same problem still remains as it requires labeled source and unlabeled target data in training time, which do not often comply with TTA settings where the model has no access to train data at test time due to privacy issues [38].

6 Discussion and conclusion

We investigate the problem of having noisy samples and the performance degradations caused by those samples in existing TTA methods. To address this issue, we propose SoTTA that is robust to noisy samples by high-confidence uniform-class sampling and entropy-sharpness minimization. Our evaluation with four noisy scenarios reveals that SoTTA outperforms state-of-the-art TTA methods in those scenarios. We believe that the takeaways from this study are a meaningful stride towards practical advances in overcoming domain shifts in test time.

Limitations and future work. While we focus on four noisy test stream scenarios, real-world test streams might have other types of sample diversities that are not considered in this work. Furthermore, recent TTA algorithms consider various temporal distribution shifts, such as temporally-correlated streams [5] and domain changes [39]. Towards developing a TTA algorithm robust to any test streams in the wild, more comprehensive and realistic considerations should be taken into account, which we believe is a meaningful future direction.

Potential negative societal impacts. As TTA requires continual computations for every test sample for adaptation, environmental concerns might be raised such as carbon emissions [37]. Recent studies such as memory-economic TTA [12] might be an effective way to mitigate this problem.

Acknowledgments and Disclosure of Funding

This work was supported by the Institute of Information & communications Technology Planning &Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2022-0-00495, On-Device Voice Phishing Call Detection).

References

Malik Boudiaf, Romain Mueller, Ismail Ben Ayed, and Luca Bertinetto. Parameter-free online test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 8344-8353, June 2022.

Dhanajit Brahma and Piyush Rai. A probabilistic framework for lifelong test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 3582-3591, June 2023.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largescale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition , pages 248-255, 2009.

Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. In International Conference on Learning Representations , 2021.

Taesik Gong, Jongheon Jeong, Taewon Kim, Yewon Kim, Jinwoo Shin, and Sung-Ju Lee. NOTE: Robust continual test-time adaptation against temporal correlation. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems , 2022.

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Commun. ACM , 63(11):139-144, oct 2020.

Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. In L. Saul, Y. Weiss, and L. Bottou, editors, Advances in Neural Information Processing Systems , volume 17. MIT Press, 2004.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2016.

Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations , 2019.

Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Representations , 2017.

Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich. Deep anomaly detection with outlier exposure. Proceedings of the International Conference on Learning Representations , 2019.

Junyuan Hong, Lingjuan Lyu, Jiayu Zhou, and Michael Spranger. MECTA: Memory-economic continual test-time model adaptation. In The Eleventh International Conference on Learning Representations , 2023.

Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Learning , volume 37 of Proceedings of Machine Learning Research , pages 448-456, Lille, France, 07-09 Jul 2015. PMLR.

Diederick P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR) , 2015.

A. Krizhevsky and G. Hinton. Learning multiple layers of features from tiny images. Master's thesis, Department of Computer Science, University of Toronto , 2009.

Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE , 86(11):2278-2324, 1998.

Dong-Hyun Lee. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML , page 896, 2013.

Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin. Training confidence-calibrated classifiers for detecting out-of-distribution samples. In International Conference on Learning Representations , 2018.

Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018.

Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of out-of-distribution image detection in neural networks. In International Conference on Learning Representations , 2018.

Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems , volume 33, pages 21464-21475. Curran Associates, Inc., 2020.

Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations (ICLR) , 2017.

TorchVision maintainers and contributors. Torchvision: Pytorch's computer vision library. https://github.com/pytorch/vision , 2016.

Yifei Ming, Ying Fan, and Yixuan Li. POEM: Out-of-distribution detection with posterior sampling. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning , volume 162 of Proceedings of Machine Learning Research , pages 15650-15665. PMLR, 17-23 Jul 2022.

Sina Mohseni, Mandar Pitale, JBS Yadawa, and Zhangyang Wang. Self-supervised learning for generalizable out-of-distribution detection. Proceedings of the AAAI Conference on Artificial Intelligence , 34(04):5216-5223, Apr. 2020.

Norman Mu and Justin Gilmer. Mnist-c: A robustness benchmark for computer vision. arXiv preprint arXiv:1906.02337 , 2019.

Zachary Nado, Shreyas Padhy, D Sculley, Alexander D'Amour, Balaji Lakshminarayanan, and Jasper Snoek. Evaluating prediction-time batch normalization for robustness under covariate shift. arXiv preprint arXiv:2006.10963 , 2020.

Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning , volume 162 of Proceedings of Machine Learning Research , pages 16888-16905. PMLR, 17-23 Jul 2022.

Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. In The Eleventh International Conference on Learning Representations , 2023.

Pau Panareda Busto and Juergen Gall. Open set domain adaptation. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) , Oct 2017.

Joaquin Quiñonero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D Lawrence. Dataset shift in machine learning . Mit Press, 2008.

Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2016.

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi, editors, Medical Image Computing and Computer-Assisted Intervention MICCAI 2015 , pages 234-241, Cham, 2015. Springer International Publishing.

Kuniaki Saito, Donghyun Kim, Stan Sclaroff, and Kate Saenko. Universal domain adaptation through self supervision. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems , volume 33, pages 16282-16292. Curran Associates, Inc., 2020.

Kuniaki Saito, Shohei Yamamoto, Yoshitaka Ushiku, and Tatsuya Harada. Open set domain adaptation by backpropagation. In Proceedings of the European Conference on Computer Vision (ECCV) , September 2018.

Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems , volume 33, pages 11539-11551. Curran Associates, Inc., 2020.

Roy Schwartz, Jesse Dodge, Noah A. Smith, and Oren Etzioni. Green ai. Commun. ACM , 63(12):54-63, nov 2020.

Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations , 2021.

Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 7201-7211, June 2022.

Tong Wu, Feiran Jia, Xiangyu Qi, Jiachen T. Wang, Vikash Sehwag, Saeed Mahloujifar, and Prateek Mittal. Uncovering adversarial risks of test-time adaptation. arXiv preprint arXiv:2301.12576 , 2023.

Jingkang Yang, Pengyun Wang, Dejian Zou, Zitang Zhou, Kunyuan Ding, WENXUAN PENG, Haoqi Wang, Guangyao Chen, Bo Li, Yiyou Sun, Xuefeng Du, Kaiyang Zhou, Wayne Zhang, Dan Hendrycks, Yixuan Li, and Ziwei Liu. Openood: Benchmarking generalized out-ofdistribution detection. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35, pages 32598-32611. Curran Associates, Inc., 2022.

Kaichao You, Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I. Jordan. Universal domain adaptation. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 2715-2724, 2019.

Qing Yu and Kiyoharu Aizawa. Unsupervised out-of-distribution detection by maximum classifier discrepancy. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , October 2019.

Longhui Yuan, Binhui Xie, and Shuang Li. Robust test-time adaptation in dynamic scenarios. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 15922-15932, June 2023.

A Experiment details

We conducted all experiments in the paper using three random seeds (0, 1, 2) and reported the average accuracies and their corresponding standard deviations. The experiments were performed on NVIDIA GeForce RTX 3090 and NVIDIA TITAN RTX GPUs. For a single execution of SoTTA, the test-time adaptation phase consumed 1 minutes for CIFAR10-C/CIFAR100-C and 10 minutes for ImageNet-C.

A.1 Baseline details

In this study, we utilized the official implementations of the baseline methods. To ensure consistency, we adopted the reported best hyperparameters documented in the respective papers or source code repositories. Furthermore, we present supplementary information regarding the implementation specifics of the baseline methods and provide a comprehensive overview of our experimental setup, including detailed descriptions of the employed hyperparameters.

SoTTA (Ours). We used ADAM optimizer [14], with a BN momentum of m = 0 . 2 , and learning rate of l = 0 . 001 with a single adaptation epoch. We set the HUS size to 64 and the confidence threshold C 0 to 0.99 for CIFAR10-C (10 classes), 0.66 for CIFAR100-C (100 classes), and 0.33 for ImageNet-C (1,000 classes). We set entropy-sharpness L2-norm constraint ρ = 0 . 5 following the suggestion [4].

PL. For PL [17], we only updated the BN layers following the previous studies [38, 39]. We set the learning rate as LR = 0 . 001 as the same as TENT [38].

TENT. For TENT [38], we set the learning rate as LR = 0 . 001 for CIFAR10-C and LR = 0 . 00025 for ImageNet-C, following the guidance provided in the original paper. We referred to the official code 3 for implementations.

LAME. LAME [1] relies on an affinity matrix and incorporates hyperparameters associated with it. We followed the hyperparameter selection specified by the authors in their paper and referred to their official code 4 for implementation details. Specifically, we employed the kNN affinity matrix with a value of k set to 5.

CoTTA. CoTTA [39] incorporates three hyperparameters: the augmentation confidence threshold p th , restoration factor p , and exponential moving average (EMA) factor m . To ensure consistency, we adopted the hyperparameter values recommended by the authors. Specifically, we set the restoration factor to p = 0 . 01 and the EMA factor to α = 0 . 999 . For the augmentation confidence threshold, the authors provide a guideline for its selection, suggesting using the 5% quantile of the softmax predictions' confidence on the source domains. We followed this guideline, which results in p th = 0 . 92 for CIFAR10-C, p th = 0 . 72 for CIFAR100-C, and p th = 0 . 1 for ImageNet-C. We referred to the official code 5 for implementing CoTTA.

EATA. For EATA [28], we followed the settings from the original paper. We set LR = 0 . 005 / 0 . 005 / 0 . 00025 for CIFAR10-C/CIFAR100-C/ImageNet-C, entropy constant E 0 = 0 . 4 × ln |Y| where |Y| is number of classes. We set cosine sample similarity threshold ϵ = 0 . 4 / 0 . 4 / 0 . 05 , trade-off parameter β = 1 / 1 / 2 , 000 , the moving average factor α = 0 . 1 . We utilized 2,000 samples for calculating Fisher importance as suggested. We referred to the official code 6 for implementing EATA.

SAR. SAR [29] aims to adapt to diverse batch sizes, and we chose a typical batch size of 64 for a fair comparison. We followed the learning rate as LR = 0 . 00025 , sharpness threshold ρ = 0 . 5 , and entropy threshold E 0 = 0 . 4 × ln |Y| where |Y| is the total number of classes, as suggested in the original paper. Finally, we froze the top layer (layer4 for ResNet18) as the original paper, and SoTTA also follows this implementation. We referred to the original code 7 for implementing SAR.

RoTTA. RoTTA [44] uses Adam Optimizer by setting learning rate as LR = 0 . 001 and β = 0 . 9 . Wefollowed the authors' hyperparameters selection from the paper, including BN-statistic exponential moving average updating rate as α = 0 . 05 , the Teacher model's exponential moving average updating rate as ν = 0 . 001 , timeliness parameter as λ t = 1 . 0 , and uncertainty parameter as λ u = 1 . 0 . We referred to the original code 8 for implementing RoTTA.

A.2 Target dataset details

CIFAR10-C/CIFAR100-C. CIFAR10-C/CIFAR100-C [9] serves as a widely adopted benchmark for evaluating the robustness of models against corruptions [27, 36, 38, 39]. Both datasets consist of 50,000 training samples and 10,000 test samples, categorized into 10/100 classes. To assess the robustness of models, datasets introduce 15 types of corruptions to the test data, including Gaussian Noise, Shot Noise, Impulse Noise, Defocus Blur, Frosted Glass Blur, Motion Blur, Zoom Blur, Snow, Frost, Fog, Brightness, Contrast, Elastic Transformation, Pixelate, and JPEG Compression. For our experiments, we adopt the highest severity level of corruption, level 5, in line with previous studies [27, 36, 38, 39]. Consequently, the datasets consist of 150,000 corrupted test samples. To train our models, we employ the ResNet18 [8] architecture as the backbone network. The model is trained on the clean training data to generate the source models. We utilize stochastic gradient descent with a momentum of 0.9 and cosine annealing learning rate scheduling [22] for 200 epochs. The initial learning rate is set to 0.1, and a batch size 128 is used during training.

ImageNet-C. ImageNet-C is another widely adopted benchmark for evaluating the robustness of models against corruptions [1, 27, 36, 38, 39]. The ImageNet dataset [3] consists of 1,281,167 training samples and 50,000 test samples. Similar to CIFAR10-C, ImageNet-C applies the same 15 types of corruptions, resulting in 750,000 corrupted test samples. We utilize the highest severity level of corruption, equivalent to CIFAR10-C. For our experiments, we employ a pre-trained ResNet18 [8] model from the TorchVision library [23], which is pre-trained on the ImageNet dataset [3] and is widely used as a backbone for various computer vision tasks.

A.3 Noisy dataset details

CIFAR100 (Near). CIFAR100 [15] consists of 50,000/10,000 training/test data with 100 classes. We utilized training data without any corruption. We undersampled the dataset to 10,000 for the CIFAR10-C and CIFAR100-C target cases by randomly removing samples and used the entire training set (50,000) for the ImageNet-C target case.

ImageNet (Near). ImageNet [3] consists of 1,281,167/50,000 training/test data with 1,000 classes. We utilized test data without any corruption. We undersampled the dataset to 10,000 for the CIFAR100-C target case by randomly removing samples.

MNIST (Far). MNIST [16] contains 60,000/10,000 training/test data with 10 classes. We utilized test data without any corruption. We used the entire test set for the CIFAR10-C/CIFAR100-C target case, and oversampled the dataset by randomly resampling, which results in 50,000 samples that is equivalent to the size of each ImageNet-C target data.

Attack. We implemented the modified indiscriminate distribution invading attack (DIA) [40]. First, we duplicated the entire set of target samples and treated them as malicious samples. Subsequently, we randomly shuffled these duplicated samples within the original target sample set. During the adaptation phase, we injected perturbations into the malicious samples to increase the overall error rate on benign samples within the same batch. As a result, we perturbed 10,000 samples (CIFAR10-C/CIFAR100-C) and 50,000 samples (ImageNet-C) to serve as attack samples. For CIFAR10-C/CIFAR100-C, we used hyperparameters of maximum perturbation constraint ϵ = 0 . 1 , attack learning rate α = 1 / 255 , and attacking steps N = 10 . For ImageNet-C, we used hyperparameters of maximum perturbation constraint ϵ = 0 . 2 , attack learning rate α = 1 / 255 , and attacking steps N = 1 .

Uniform random noise (Noise). We generated a uniform random valued image in the scaled RGB range [0 , 1] , with the same height and width as the corresponding target dataset. We generated the same amount of noise samples as each target dataset.

B Result details

C Additional ablative studies

We conducted experiments to understand the sensitivity of our two hyperparameters: confidence threshold ( C 0 ) and BN momentum ( m ). We varied C 0 and m and reported the corresponding accuracy.







































(a) Effect of hyperparameter

C

0

.







































(b) Effect of hyperparameter

m

.

Confidence threshold. Our result shows that the selection of C 0 shows similar patterns across different scenarios (Benign ∼ Noise). The result illustrates a tradeoff; a low C 0 value does not effectively reject noisy samples, while a high C 0 value filters benign data. We found a proper value of C 0 (0.99) that generally works well across the scenarios. Also, we found that the optimal C 0 depends primarily on in-distribution data. Our interpretation is that setting different C 0 values for CIFAR10-C, CIFAR100-C, and ImageNet-C is straightforward as they have a different number of classes (10, 100, and 1,000), which leads to different ranges of the model's confidence.

BN momentum. Across the tested range, the variations in performance were found to be negligible. This finding indicates that choosing a low momentum value from within the specified range ([0.05, 0.3]) is adequate to maintain a favorable performance. Please note that setting a high momentum would corrupt the result, which is implicated by the algorithms directly utilizing test-time statistics (e.g., TENT) suffering from accuracy degradation with noisy data streams (e.g., TENT: 81.0% → 52.1% for Noise at Table 1).

D Further discussions

D.1 Theoretical explanation of the impact of noisy data streams

We provide a theoretical explanation of the impact of noisy data streams with a common entropy minimization as an example. With the Bayesian-learning-based frameworks [2, 7], we can express the posterior distribution p of the model in terms of training data D and benign test data B in test-time adaptation:

The posterior distribution of model parameters depends on the prior distribution q and the average of entropy H of benign samples with a multiplier λ . Here, we incorporate the additional noisy data stream N into Equation 7 and introduce a new posterior distribution considering noisy streams:

With Equation 7 and Equation 8, we can now derive model parameter variations caused by noisy test samples:

Equation 9 implies that the (1) model adapted only from benign data and (2) model adapted with both benign and noisy data differ by the amount of the average entropy of noisy samples. This also suggests that a high entropy from severe noisy samples would result in a significant model drift in adaptation (i.e., model corruption).

D.2 Comparison with previous TTA methods

D.2.1 EATA and SAR

While SoTTA, EATA [28], and SAR [29] all leverage sample filtering strategy, the key distinction of input-wise robustness of SoTTA and EATA/SAR lies in three aspects: (1) Our high-confidence sampling strategy in SoTTA aims to filter noisy samples by utilizing only the samples with high confidence, while both EATA and SAR use a different approach that excludes a few high-entropy samples, particularly during the early adaptation stage. In our preliminary study, we found that our method excludes 99.98% of the noisy samples, whereas EATA and SAR exclude 33.55% of such samples. (2) While EATA and SAR adapt to every incoming low-entropy sample, SoTTA leverages a uniform-class memory management approach to prevent overfitting. As shown in Figure 5b, noisy samples often lead to imbalanced class predictions, and these skewed distributions could lead to an undesirable bias in p ( y ) and thus might negatively impact TTA objectives, such as entropy minimization. The ablation study in Table 10 shows the effectiveness of uniform sampling with a 3.9%p accuracy improvement. (3) EATA and SAR reset the memory buffer and restart the sample collection process for each adaptation. This strategy is susceptible to overfitting due to a smaller number of samples used for adaptation and the temporal distribution drift of the samples. In contrast, our continual memory management approach effectively mitigates this issue by retaining highconfidence uniform-class samples in the memory, as shown in Table 10.

We acknowledge that both SoTTA and SAR utilize sharpness-aware minimization proposed by Foret et al. [4]. However, we clarify that the motivation behind using SAM is different. While SAR intends to avoid model collapse when exposed to samples with large gradients, we aim to enhance the model's robustness to noisy samples with high confidence scores. As illustrated in Figure 6, we observed that entropy-sharpness minimization effectively prevents the model from overfitting to noisy samples. As a result, while our algorithm led to marginal performance degradation in noisy settings (82.2% → 80.0% for Noise), EATA and SAR showed significant degradation (EATA 82.4% → 36.0% for Noise; SAR 78.3% → 58.3% for Noise).

D.2.2 RoTTA

Regarding our high-confidence uniform sampling technique, RoTTA [44] could be compared. First of all, RoTTA's objective is different from ours; RoTTA focused on temporal distribution changes of test streams without considering noisy samples. Similar to SoTTA, RoTTA's memory bank maintains recent high-confidence samples. However, RoTTA has no filtering mechanism for low-confidence samples, which makes RoTTA fail to avoid noisy samples, especially in the early stage of TTA. In contrast, our confidence-based memory management scheme effectively rejects noisy samples, and

























































thus it prevents potential model drift from the beginning of TTA scenarios. As a result, our approach outperforms RoTTA in noisy test streams (e.g., 5.4%p better than RoTTA on CIFAR10-C).

D.3 Comparison with out-of-distribution detection algorithms

We discussed the limitation of applying out-of-distribution detection to TTA in Section 5. Still, we are curious about the effect of applying out-of-distribution algorithms to our scenario. To this end, we conduct experiments using one of the out-of-distribution algorithms, ODIN [20], in our noisy data streams. Specifically, we filtered OOD samples detected by ODIN and performed TTA algorithms on the samples left.

Note that similar to prior studies on OOD, ODIN uses a thresholding approach to predict whether a sample is OOD. It thus requires validation data with binary labels indicating whether it is indistribution or OOD to decide the best threshold δ . However, in TTA scenarios, validation data is not provided, which makes it difficult to apply OOD algorithms directly in our scenario. We circumvented this problem using the labeled test batches to get the best threshold. Following the original paper, we searched for the best threshold from 0.1 to 0.12 with a step size of 0.000001, which took over 20,000 times longer than the original TTA algorithm.

Table 11 shows that the impact of discarding OOD samples with ODIN is negligible, yielding only a 0.3%p improvement in the average accuracy despite a huge computation cost. Also, Figure 9 shows the high sensitivity of ODIN with respect to threshold hyperparameter δ , which implies that applying OOD in TTA is impractical.

We conclude the practical limitations of OOD detection algorithms for TTA as follows: (1) OOD methods assume that a model is fixed during test time, while a model changes continually in TTA. (2) As previously noted, most OOD algorithms require labels for validation data unavailable in TTA scenarios. Even using the same test dataset for selecting the threshold, the performance improvement was marginal. (3) Low performance possibly results from the fact that OOD detection studies are

built on the condition that training and test domains are the same, which differs from TTA's scenario. These collectively make it difficult to apply OOD detection studies directly to TTA scenarios.

D.4 Applying to other domains

While this study primarily focuses on classification tasks, there are other tasks where test-time adaptation would be useful. Here we discuss the applicability of SoTTA to (1) image segmentation and (2) object detection, which are crucial in autonomous driving scenarios.

For image segmentation, when noisy objects are present in the input, the model might produce noisy predictions on those pixels, leading to detrimental results. Extending SoTTA to operate at the pixel level would allow it to be compatible with the segmentation task while minimizing the negative influences of those noisy pixels on model predictions in test-time adaptation scenarios.

Similarly, SoTTA could be tailored to object detection's classification (recognition) task. For example, in the context of the YOLO framework [32], SoTTA could filter and store grids with high confidence for test-time adaptation, enhancing detection accuracy. However, our current approach must address the localization task (bounding box regression) during test-time adaptation. Implementing this feature is non-trivial and would require careful consideration and potential redesign of certain aspects of our methodology. Accurately localizing bounding boxes during test-time adaptation presents an exciting avenue for future research.

E License of assets

Datasets CIFAR10/CIFAR100 (MIT License), CIFAR10-C/CIFAR100-C (Creative Commons Attribution 4.0 International), ImageNet-C (Apache 2.0), and MNIST (CC-BY-NC-SA 3.0).

Codes Torchvision for ResNet18 (Apache 2.0), the official repository of CoTTA (MIT License), the official repository of TENT (MIT License), the official repository of LAME (CC BY-NC-SA 4.0), the official repository of EATA (MIT License), the official repository of SAR (BSD 3-Clause License), and the official repository of RoTTA (MIT License).

Peer reviews:
comments: SUMMARY:
The authors point out that model may suffer from non-interest samples while TTA. Existing TTA methods are not robust to these samples. To address these issues, the authors proposed a methods called SoTTA with two key components, input-wise robustness via high-confidence uniform-class sampling and parameter-wise robustness via entropy-sharpness minimization.

STRENGTHS:
The authors focus on TTA in the wild and point out that TTA with non-interest would lead to performance degradation.

WEAKNESSES:
The proposed method seems to be not novel.

A) The proposed “Input-wise robustness via high-confidence uniform-class sampling” method excludes low-confidence samples in TTA. Such samples filtration strategies have been explored in ETA [Efficient Test-Time Model Adaptation without Forgetting].

B) The proposed “Parameter-wise robustness via entropy-sharpness minimization” method introduces entropy-sharpness minimization, which have been explored in SAR [Towards Stable Test-time Adaptation in Dynamic Wild World].

It would be better for the authors to clarify the differences between the proposed methods and the TTA methods I mentioned above.

In the experimental results, I found the proposed always outperforms SAR. But in my understanding, the main components of these two methods are almost the same. Both of them include low-confidence sample filtration strategy and entropy-sharpness minimization. So I have no idea why the proposed method can yield much better results.

QUESTIONS:
See comments above.

LIMITATIONS:
See comments above.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-06-29

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies a practical problem of test-time adaptation where non-interest testing samples may appear and mislead the adaptation. This problem is quite serious in practical applications, and the problem setting is relatively novel. To address this problem, the authors propose the SoTTA method, which solves this problem in two aspects. For input-wise robustness, SoTTA filters out the non-interest samples with a high-confidence uniform-class memory buffer. For parameter-wise robustness, entropy-sharpness minimization is adopted to ensure that the landscape is smooth during the adaptation. The proposed method is evaluated on two benchmark datasets under one standard setting and four robust settings with different types of non-interest samples. The results show that SoTTA gives state-of-the-art performance compared to existing TTA methods.\n\nSTRENGTHS:\n1. This paper studies a novel problem setting of test-time adaptation, where non-interest testing samples may appear and mislead the adaptation. This is a relatively novel and practical problem in real applications.\n2. The overall idea of this paper makes sense. This paper tackles the harmful impact of non-interest with a filter and robust loss function. The experiments in the main paper also show that the proposed method gives state-of-the-art performance.\n\nWEAKNESSES:\n1. First, this paper ignores one common TTA benchmark, i.e., CIFAR100. Additionally, the results of ImageNet-C have been hidden in the appendix. These key results should be presented in the main paper with comprehensive analysis.\n2. One TTA algorithm, EATA [1], adopts a similar sample selection strategy during adaptation. This method should be taken into consideration in the experiment parts because it has the potential to handle the problem of non-interest samples.\n3. This paper does not provide any discussion about the selection and sensitivity of threshold C0. Intuitively, the optimal C0 depends on both in-distribution data and out-of-distribution data, and it is not easy to decide in real applications.\n4. There is more related work that can be discussed to further improve this paper. For test-time adaptation, the above-mentioned EATA method [1] and recent advanced TTA methods [2-4] about robustness and evaluation should be considered. For out-of-distribution detection, methods [5-6] that can be efficiently optimized in an unsupervised manner should be taken into consideration.\n\n**Reference**\n\n[1] Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, Mingkui Tan: Efficient Test-Time Model Adaptation without Forgetting. ICML 2022: 16888-16905\n\n[2] Hao Zhao, Yuejiang Liu, Alexandre Alahi, Tao Lin: On Pitfalls of Test-Time Adaptation. ICML 2023\n\n[3] Zhi Zhou, Lan-Zhe Guo, Lin-Han Jia, Dingchu Zhang, Yu-Feng Li: ODS: Test-Time Adaptation in the Presence of Open-World Data Shift. ICML 2023\n\n[4] Tong Wu, Feiran Jia, Xiangyu Qi, Jiachen T. Wang, Vikash Sehwag, Saeed Mahloujifar, Prateek Mittal: Uncovering Adversarial Risks of Test-Time Adaptation. ICML 2023\n\n[5] Zhi Zhou, Lan-Zhe Guo, Zhanzhan Cheng, Yu-Feng Li, Shiliang Pu: STEP: Out-of-Distribution Detection in the Presence of Limited In-Distribution Labeled Data. NeurIPS 2021: 29168-29180\n\n[6] Jiangpeng He, Fengqing Zhu: Out-Of-Distribution Detection In Unsupervised Continual Learning. CVPR Workshops 2022: 3849-3854\n\nQUESTIONS:\n1. The experiment shows that RoTTA outperforms existing TTA methods even in standard settings. Can the authors explain why this phenomenon occurs? Intuitively, the techniques mentioned in this article are all aimed at addressing the problem of non-interesting samples. What factors enable them to achieve better performance when non-interesting samples do not exist?\n2. Can the authors provide some sensitivity analysis regarding the threshold C0? It is very important for the actual effect of filtering out non-interesting samples. An adaptive scheme for setting C0 would also be acceptable.\n3. One naive way to solve the proposed setting is to adopt an out-of-distribution process before test-time adaptation. How does this naive method work in the proposed setting?\n\n**[IMPORTANT]** I would like to raise my score if you successfully address my concerns in the Weakness and Question sections. Overall, this paper is interesting and I appreciate it. However, it is currently slightly below the level of acceptance.\n\nLIMITATIONS:\nThe authors have properly discussed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 1, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis article presents a new Test-Time Adaptation (TTA) scenario, wherein the model is adapted to noisy test streams. To address the challenges posed by this scenario, the paper introduces the Screening-out Test-Time Adaptation (SoTTA) algorithm, which leverages input-wise and parameter-wise robustness. The effectiveness of SoTTA is demonstrated through extensive comparison experiments conducted on TTA benchmarks.\n\nSTRENGTHS:\n1. This work proposes a novel test time adaptation setup, which considers the noisy data (non-interest) in real world application during test phase, whose motivation is convincing.\n2. The significant performance gain shows SoTTA addresses the challenges under noisy test stream well.\n\nWEAKNESSES:\n1. The initial component of the method is inspired by the memory bank concept introduced in RoTTA, albeit in a simplified form. The second component, pioneered by SAR[1], represents a novel approach within the TTA domain. However, it is worth noting that the methodology section may lack originality.\n2. The sensitivity analysis and selection criterion for key hyperparameters, specifically $m,C_0$ , are not included in the current study.\n3. The motivation of this paper stems primarily from experimental findings, and it would be advantageous to analyze the motivation or methods from a theoretical perspective.\n4. The paper contains several typographical errors, and it would be valuable to thoroughly revise it.\n\n[1] Towards stable test-time adaptation in dynamic wild world\n\nQUESTIONS:\n1. What would be the outcome if CSTU and ESM were combined? Would the performance surpass that of SoTTA?\n2. It is acknowledged that in real-world scenarios, the non-interest scenes mentioned in the experimental section may occur simultaneously. Have any relevant experiments been conducted to address this?\n\nLIMITATIONS:\nActually, the problem setting is interesting to some extent, however, the techniques in this paper are similar to several compared methods, which may limit the novelty of this paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

comments: SUMMARY:
This paper proposes screening-out test-time adaptation which is claimed robust to non-interest samples. It filters out the impact of non-interest samples with a high-confidence uniform-class sampling. It proposes entropy-sharpness minimization to deal with large gradients. Experiments are completed on CIFAR10-C and ImageNet-C.

STRENGTHS:
1. The proposed problem (screening-out test-time adaptation) is novel and sharp.
2. This paper has a good presentation. For example, Fig.1 and Fig.2 are very clear.
3. Entropy-sharpness minimization seems reasonable as discussed in Section 3.2.
4. Experiments are completed on both small and large datasets including ImageNet-C. As shown in Tab.1, the improvement is good enough.

WEAKNESSES:
1. The problem is interesting but the proposed two methods are straightforward. I am a little worried about the technical novelty.
2. The design of input-wise robustness is hard coding. Although some samples are bad for optimization, they may be essential for the perception.
3. Only classification is completed. However, segmentation is also very important in autonomous driving scenarios.

QUESTIONS:
1. Can it also work on segmentation and detection? Please discuss.

LIMITATIONS:
Please refer to "weakness".
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06