T EST -T IME A DAPTATION VIA S ELF -T RAINING WITH N EAREST N EIGHBOR I NFORMATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Test-time adaptation (TTA) aims to adapt a trained classifier using online unlabeled test data only, without any information related to the training procedure.
Most existing TTA methods adapt the trained classifier using the classifier’s prediction on the test data as pseudo-label.
However, under test-time domain shift, accuracy of the pseudo labels cannot be guaranteed, and thus the TTA methods often encounter performance degradation at the adapted classifier.
To overcome this limitation, we propose a novel test-time adaptation method, called Test-time Adaptation via Self-Training with nearest neighbor information (TAST) , which is composed of the following procedures: (1) adds trainable adaptation modules on top of the trained feature extractor; (2) newly defines a pseudo-label distribution for the test data by using the nearest neighbor information; (3) trains these modules only a few times during test time to match the nearest neighbor-based pseudo label distribution and a prototype-based class distribution for the test data; and (4) predicts the label of test data using the average predicted class distribution from these modules.
The pseudo-label generation is based on the basic intuition that a test data and its nearest neighbor in the embedding space are likely to share the same label under the domain shift.
By utilizing multiple randomly initialized adaptation modules, TAST extracts useful information for the classification of the test data under the domain shift, using the nearest neighbor information.
Our experiments on two standard benchmark tasks, domain generalization, namely VLCS, PACS, OfficeHome, and TerraIncognita, and image corruption, particularly CIFAR-10/100C, show that TAST outperforms the state-of-the-art TTA methods. Our implementation is available in the supplementary material.
I NTRODUCTION
Deep neural networks often encounter significant performance degradations under domain shift (i.e., distribution shift).
This phenomenon has been observed in various tasks including classification (Taori et al., 2020; Wang et al., 2021b), visual recognition (Saenko et al., 2010; Csurka, 2017), and reinforcement learning (Cobbe et al., 2019; Mendonca et al., 2020; Lee and Chung, 2021b).
There are two broad classes of domain adaptation methods that attempt to solve this problem: supervised domain adaptation (SDA) (Tzeng et al., 2015; Motiian et al., 2017) and unsupervised domain adaptation (UDA) (Ganin and Lempitsky, 2015;
Long et al., 2016; Sener et al., 2016).
Both SDA and UDA methods aim to obtain domain-invariant representations by aligning the representations of training and test data closely in the embedding space.
While testing, UDA methods require the training dataset and SDA methods additionally require labeled data of the test domain.
However, in practice, it is often difficult to access training datasets or labeled data in the test domain during test time, due to data security or labeling cost.
Test-time adaptation (TTA) (Iwasawa and Matsuo, 2021; Wang et al., 2021a) is a prominent approach to alleviate the problems caused by the domain shift.
TTA methods aim to adapt the trained model to the test domain without a labeled dataset in the test domain and any information related to the training procedure (e.g., training dataset, feature statistics of training domain (Sun et al., 2020;
Liu et al., 2021; Eastwood et al., 2022)).
TTA methods have access to the online unlabeled test data only, whereas domain adaptation methods assume access to the whole (i.e., offline) test data.
There are three popular categories for TTA: normalization-based (Schneider et al., 2020), entropy minimization (Liang et al., 2020; Wang et al., 2021a) and prototype-based methods (Iwasawa and Matsuo, 2021).
Normalization method replaces the batch normalization (BN) statistics of the trained model with the BN statistics estimated on test data, and does not update model parameters except for the BN layers.
Entropy minimization methods fine-tune the trained feature extractor, which is the trained classifier except the last linear layer, by minimizing the prediction entropy of test data.
These methods force the classifier to have over-confident predictions for the test data, and thus have a risk of degrading model calibration (Guo et al., 2017; Mukhoti et al., 2020), a measure of model interpretability and reliability.
One form of entropy minimization is self-training (Rosenberg et al., 2005; Lee, 2013; Xie et al., 2020).
Self-training methods use predictions from the classifier as pseudo labels for the test data and fine-tune the classifier to make it fit to the pseudo labels.
These methods have a limitation that the fine-tuned classifier can overfit to the inaccurate pseudo labels, resulting in confirmation bias (Arazo et al., 2020).
This limitation can be harmful when the performance of the trained classifier is significantly degraded due to domain shift.
On the other hand,
Iwasawa and Matsuo (2021) proposed a prototype-based TTA method, named T3A, that simply modifies a trained linear classifier (the last layer) by using the pseudo-prototype representations of each class and the prototype-based classification for test data, where the prototypes are constructed by previous test data and the prediction for the data from trained classifier.
T3A does not update the trained feature extractor at test time.
T3A is simple but it brings a marginal performance gain (Table 1 and 3).
In this work, we propose a new test-time adaptation method, which is simple yet effective in mitigating the confirmation bias problem of self-training, by adding adaptation modules on top of the feature extractor, which are simply trainable during test time.
We use the prototype-based classifier as in T3A, but not in the embedding space of the original feature extractor but in the embedding space of the adaptation modules, trained with nearest neighbor information, to achieve higher performance gains than the original simple prototype-based classifier method.
Our method, named
Test-time Adaptation via Self-Training with nearest neighbor information (TAST) , is composed of the following procedures: (1) adds randomly initialized adaptation modules on top of the feature extractor at the beginning of test time (Figure 1); (2) generates pseudo label distribution for a test data considering the nearest neighbor information; (3) trains the adaptation modules only a few times during test time to match the nearest neighbor-based pseudo label distribution and a prototype-based class distribution for the test data; and (4) predicts the label of test data using the average predicted class distribution from the adaptation modules.
Specifically, in (1), we add the trainable adaptation modules to obtain new feature embeddings that are useful for classification in the test domain.
In (2), TAST assigns the mean of the labels of the nearby examples in the embedding space as the pseudo label distribution for the test data based on the idea that a test data and its nearest neighbors are more likely to have the same label.
In (3), TAST trains the adaptation modules to output the pseudo label distribution when the test data is fed into (Figure 1 Right ).
And in (4), we average the predicted class distributions from adaptation modules for the prediction of test data (Figure 1 Left ).
We investigate the effectiveness of TAST on two standard benchmarks, domain generalization and image corruption.
We demonstrate that TAST outperforms the current state-of-the-art test-time adaptation methods such as Tent (Wang et al., 2021a), T3A, and TTT++ (Liu et al., 2021) on the two benchmarks.
For example, TAST surpasses the current state-of-the-art algorithm by 1 .
01% on average with ResNet-18 learned by Empirical Risk Minimization (ERM) on the domain generalization benchmarks.
Extensive ablation studies show that both the nearest neighbor information and the adaptation module utilization contribute to the performance increase.
Moreover, we experimentally found that the adaptation modules adapt feature extractor outputs effectively although the adaptation modules are randomly initialized at the beginning of test time and trained with a few gradient steps per test batch during test time.
P RELIMINARIES
Test-time domain shift Consider a labeled dataset D train = { ( x i , y i ) } n train i =1 drawn from a distribution P train , where x ∈ R d and y ∈ Y :
= { 1 , 2 , · · · , K } for a K -class classification problem.
A number of classifiers have been proposed that easily classify unseen test data under the i.i.d.
assumption that unseen test data D test is drawn from the same distribution as training data, i.e.,
P train =
P test .
We assume the classifier is a deep neural network composed of two parts: a feature
[Figure]
extractor f θ : R d → R d z and a linear classifier g w : R d z → Y , where θ and w are network parameters.
ERM optimizes θ and w to obtain a good classifier for future samples in D test by minimizing the objective function L ( θ, w )
=
E ( x,y ) ∈
D train [ l ( g w ( f θ ( x )) , y )] , where l is a loss function such as cross-entropy loss.
However, under the test-time domain shift (i.e., distribution shift), the i.i.d.
assumption between the training and test distributions does not hold, i.e., P train ̸
= P test , and the trained classifiers often show poor classification performance for the test data.
Prototype-based classification in test-time adaptation Prototype-based classification refers to a method that obtains prototype representations, which represent each class in the embedding space, and then predicts the label of an input as the class of the nearest prototype.
Since labeled data is not available in the TTA setting, T3A (Iwasawa and Matsuo, 2021) utilizes a support set that is composed of previous test data and their predictions for the test data by the trained classifier.
T3A does not modify parameters of the classifier.
Since the embedding space of the feature extractor is unchanged during the test time, T3A constructs the support set using the feature representations for test data instead of the data itself.
Specifically, a support set S t = { S 1 t , S 2 t , · · · , S Kt } is a set of test samples until time t .
The support set is initialized with the weight of the last linear classifier, i.e., S k 0 =
(cid:110) w k ∥ w k ∥ (cid:111) , where w k is the parts of w related to k -th class for k = 1 , 2 , . . .
, K .
At time t , the support set is updated as
[Equation]
where p k represents the likelihood that the classifier assigns x t to the k -th class.
Using the support set S kt , one can obtain the class prototype for class k by taking the centroid of the representations in the support set.
Formally, the prototype µ k for class k is computed as µ k = 1 | S kt | (cid:80) z ∈ S kt z for k = 1 , 2 , · · · , K .
Then, the prediction for an input x t is made by comparing the distances between the embedding of x t and the prototypes,
i.e., ˆ y = arg
min c d ( f θ ( x t ) , µ c ) with a pre-defined metric d such as Euclidean distance or cosine similarity.
Since the wrongly pseudo labeled examples can degrade the classification performance, the support examples with unconfident pseudo labels are regarded as unreliable examples and filtered out, i.e., at time stamp t ,
[Equation]
where α k is the M -th largest prediction entropy of the samples from S kt , H is Shannon entropy (Lin, 1991), and σ is the softmax function.
T3A modifies only the support set configuration and does not update the trained model parameters at test time.
Thus, T3A cannot effectively mitigate the classification performance degradation caused by test-time domain shift.
To address this issue, we extract useful information for classification of the test data by utilizing multiple randomly initialized adaptation modules that are trained using nearest neighbor-based pseudo labels.
M ETHODOLOGY
In this section, we describe two main components of our method TAST: adaptation module utilization (Section 3.1) and pseudo-label generation considering nearest neighbor information (Section 3.2).
3. A DAPTATION MODULE
We first discuss the parts to be fine-tuned in the trained classifier before explaining our test-time adaptation method.
One possible choice is to fine-tune the whole network parameters in the classifier during test time, but this approach can be unstable and inefficient (Wang et al., 2021a; Kumar et al., 2022).
Another choice is to fine-tune only the parameters of batch normalization (BN) layers in the classifier as in Wang et al.
(2021a).
Although it achieves effective test-time adaptation, it has a limitation that it can be utilized only if there are BN layers in the trained classifier.
The other choice is to train a new classifier added on top of the frozen feature extractor during test time as in
Lee and Chung (2021a).
We construct the new classifier by adding a randomly initialized adaptation module as illustrated in Figure 1.
During the test time, we train the adaptation module and predict the label of the test data using prototype-based class distributions from the adaptation module. The random initialization of the adaptation module may cause performance degradation of trained classifier.
Thus, we consider an ensemble scheme (Wen et al., 2020; YM.
et al., 2020; Mesbah et al., 2021) to alleviate the issues caused by the random initialization of the adaptation modules to obtain more robust and accurate predictions.
We train the adaptation modules independently and predict the label of the test data using the average predicted class distribution from the adaptation modules.
3. S ELF - TRAINING WITH NEAREST NEIGHBOR INFORMATION
TAST generates pseudo label distributions for unlabeled test data with the nearest neighbor information and fine-tunes the adaptation modules with the pseudo label distributions.
The whole adaptation procedure of TAST is described in Algorithm 1.
We first update the support set S and filter out the unconfident examples from the support set as in Iwasawa and Matsuo (2021) with the equations (1) and (2).
Then, we find N s nearby support examples of test data x in the embedding space of f θ .
We denote N ( x ; S ) as the set of nearby support examples of x ,
[Equation]
where β x is the distance between x and the N s -th nearest neighbor of x from S in the embedding space of f θ .
Each adaptation module is trained individually during test time.
For the i -th adaptation module h ϕ 2 , we compute the prototype representations µ i, 1 , µ i, 2 , . .
. , µ i,K in the embedding space of h ϕ i ◦ f θ with a support set S = { S 1 , S 2 , · · · , S K } , i.e., µ i,k = 1 | S k | (cid:80) z [Equation]
S k h ϕ
i ( z ) , for k = 1 , 2 , . . .
, K .
With the prototypes, we compute the prototype-based predicted class distribution of the nearby support examples in the embedding space of h ϕ i ◦ f θ , i.e., for z ∈ N ( x ; S ) , the likelihood that the prototype-based classifier assigns z to the k -th class is computed as [Equation] proto
[Equation]
where τ is the softmax temperature 3 .
With the nearest neighbor information, TAST generates a pseudo label distribution ˆ p TAST i of x by aggregating prototype-based predicted class distribution of the nearby support examples in N ( x ; S ) as
[Equation]
for k = 1 , 2 , . . .
, K .
Specifically, we use the one-hot class distributions for pseudo label generation as in Lee (2013); Sohn et al.
Then, we fine-tune the adaptation modules by minimizing the cross-entropy loss between the predicted class distribution of the test example and the nearest neighbor-based pseudo label distribution:
[Equation]
where CE denotes the standard cross-entropy loss.
We iterate the pseudo labeling and fine-tuning processes for T steps per batch.
We note that our method does not propagate gradients into the pseudo labels as in Laine and Aila (2017); Berthelot et al.
Finally, we predict the label of x using the average predicted class distribution p TAST i from the adaptation modules, i.e.,
[Equation]
TAST [Equation] On the other hand, we consider a variant of TAST, named TAST-BN, that fine-tunes the BN layers instead of adaptation modules.
The support set stores the test data itself instead of the feature representations since the embedding space of the feature extractor steadily changes during the test time.
The pseudocode for TAST-BN is presented in Appendix B.
E XPERIMENTS
In this section, we show the effectiveness of our method compared to the state-of-the-art test-time adaptation methods on two standard benchmarks, i.e., domain generalization and image corruption.
[Table] We compare TAST with the following baseline methods: (1) Pseudo Labeling (PL) (Lee, 2013) finetunes the trained classifier using confident pseudo labels based on classifier predictions; (2) PLClf is a modified version of PL that fine-tunes only the last linear classifier; (3) Tent (Wang et al., 2021a) fine-tunes only the parameters of the BN layers to minimize the prediction entropy of test data; (4) TentAdapter is a modified version of Tent that adds a BN layer between the feature extractor and the last linear classifier, and fine-tunes only the added BN layer; (5) TentClf is a modified version of Tent that fine-tunes only the last linear classifier instead of the BN layers; (6) SHOTIM (Liang et al., 2020) updates the feature extractor to maximize the mutual information between an input and its prediction; (7) SHOT is a method that adds a pseudo-label loss to SHOTIM; (8) T3A predicts the label of the test data by comparing distances between test data and the generated pseudoprototypes.
Originally, SHOT is one of source-free domain adaptation methods which focus on the offline setting, but we compare our method with the online version of SHOT for a fair comparison.
4. D OMAIN G ENERALIZATION
The domain generalization benchmarks are designed to evaluate the generalization ability of the trained classifiers to the unseen domain.
The evaluation is performed by a leave-one-domain-out procedure, which uses a domain as a test domain and the remaining domains as training domains.
We use the publicly released code 4 of T3A for the domain generalization benchmarks.
4.1. E XPERIMENTAL SETUP
Datasets We test TAST on four domain generalization benchmarks, specifically VLCS (Fang et al., 2013), PACS (Li et al., 2017), OfficeHome (Venkateswara et al., 2017), and TerraIncognita (Beery et al., 2018).
VLCS is composed of photographic images from four different datasets (PASCAL VOC207 (Everingham et al., 2010), LableMe (Russell et al., 2008), Caltech 101 (Fei-Fei et al., 2007), and SUN09 (Choi et al., 2010)), consisting of 10,729 examples of 5 categories (bird, car, chair, dog, and person).
PACS is composed of images of objects from four different domains (photo, art, cartoon, and sketch), consisting of 9,991 examples of 7 categories (dog, elephant, giraffe, guitar, horse, house, and person).
OfficeHome is composed of images of objects in the office and home from 4 different domains (artistic images, clip art, product, and real-world images), consisting of 15,588 examples of 65 categories (e.g., alarm clock, backpack, and batteries).
TerraIncognita is composed of wild animal images taken from 4 different locations (L100, L38, L43, and L46), consisting of 24,788 examples of 10 classes.
Training setup For a fair comparison, we follow the training setup including dataset splits and hyperparameter selection method used in T3A. We use residual networks (He et al., 2016) including batch normalization layers with 18 and 50 layers (hereinafter referred to as ResNet-18 and ResNet50, respectively), which are widely used for classification tasks.
We train the networks with various learning algorithms such as ERM and CORAL (Sun and Saenko, 2016).
Details about the learning algorithms are explained in Appendix A.
The backbone networks are trained with the default hyperparameters introduced in Gulrajani and Lopez-Paz (2021).
We use a BatchEnsemble (Wen et al., 2020), which is an efficient ensemble method that reduces the computational cost by weightsharing, for the adaptation modules of TAST.
The output dimension of each adaptation module is set to a quarter of the output dimension of the feature extractor 5 , e.g., 128 for ResNet-18.
We use Kaiming normalization (He et al., 2015) for initializing the adaptation modules at the beginning of test time.
We run experiments using four different random seeds.
More details on the training setups can be found in Appendix A. Moreover, a discussion on computation complexity such as runtime comparison is summarized in Appendix A.
Hyperparameters For a fair comparison, the baseline methods use the same hyperparameters as in Iwasawa and Matsuo (2021).
TAST uses the same set of possible values for each hyperparameter with baseline methods.
TAST involves four hyperparameters: the number of gradient steps per adaptation T , the number of support examples per each class M , and the number of nearby support examples N s , the number of adaptation modules N e .
We define a finite set of possible values for each hyperparameter, N s ∈ { 1 , 2 , 4 , 8 } , T ∈ { 1 , 3 } , and M ∈ { 1 , 5 , 20 , 50 , 100 , − 1 } , where −means to storing all samples without filtering.
N e is set to 20 .
We use Adam optimizer with a learning rate of 0.001.
More details on the hyperparameters can be found in Appendix A. Moreover, refer to Appendix C for the sensitivity analysis on hyperparameters including the test batch sizes.
4.1. E XPERIMENTAL RESULTS
In Table 1, we summarize the experimental results of test-time adaptation methods using classifiers trained by ERM.
Our method consistently improves the performance of the trained classifiers by 2 .
17% for ResNet-18 and 1 .
21% for ResNet-50 on average, respectively.
TAST also outperforms the baseline methods including the state-of-the-art test-time adaptation method T3A. Compared to T3A, TAST shows better performance by 1 .
01% for ResNet-18 and 0 .
69% for ResNet-50 on average, respectively.
Especially, we find that our method significantly improves the performance of the trained classifiers in the TerraIncognita benchmark, which is a challenging benchmark in that the trained classifier shows the lowest prediction accuracy.
We observe that the performance of the baseline methods, which fine-tune the feature extractors, is lower than that of the classifiers without adaptation, whereas TAST-BN improves the performance of the trained classifiers.
Refer to
Appendix C for the experimental results of test-time adaptation methods using classifiers trained by different learning algorithms such as CORAL (Sun and Saenko, 2016) and MMD (Li et al., 2018).
Effect of nearest neighbor information To understand the effect of nearest neighbor information, we compare Tent and TAST-BN, both of which fine-tine the BN layers.
To adjust the BN layers,
Tent uses entropy minimization loss, whereas TAST-BN uses the pseudo-label loss using the nearest neighbor information.
As shown in Table 1, the performances of TAST-BN is better than those of
Tent by 3 . 3% for ResNet-18 and 2 . 89% for ResNet-50, respectively.
In addition, we consider an ablated variant of TAST, named TAST-N, that removes adaptation modules from TAST.
TAST-N is optimization-free and has the same support set configuration as T3A. T3A uses the prototype-based prediction of the test data itself, whereas TAST-N uses the aggregated predicted class distribution [Table] of the nearby support examples.
As shown in Table [Table] the prediction using the nearest neighbor information leads to a performance gain of 0 .
43% on average.
Effect of adaptation modules TAST adds randomly initialized adaptation modules on top of the trained feature extractor as illustrated in Figure 1 and trains the adaptation modules during test time.
For each test batch, we update the adaptation modules T times using pseudo label distributions considering nearest neighbor information.
We set T to 1 or 3 throughout all experiments.
To verify that the few step updates are sufficient to train the adaptation modules, we conduct experiments with different T ∈ { 0 , 1 , 2 , 4 , 8 } .
We test on PACS using classifiers learned by ERM while M and N s are set to − 1 and one of { 1 , 2 , 4 , 8 } .
We summarize the experimental results in Figure 2.
We observe that the performance of the adapted classifier is better than that of the non-adapted classifier (i.e.,
T = 0 ) and robust to changes in T .
Hence, we conjecture that we can obtain a sufficiently good adaptation module with a few-step updates similar to Lee and Chung (2021a).
In addition, to investigate the effect of adaptation modules, we test TAST with a varying number of adaptation modules, e.g., N e ∈ { 1 , 5 , 10 , 20 } .
In Table 2, we find that utilizing a single adaptation module leads to degraded performance than TAST-N.
However, TAST with multiple adaptation modules shows improvement over TAST-N and T3A on average.
[Figure]
[Table]
4. I MAGE C ORRUPTION
The image corruption benchmark to evaluate the robustness of a classifier to an unseen test domain when the classifier a training domain
4.2. E XPERIMENTAL SETUP
We test robustness of TAST to image corruption on CIFAR-10/100 (Krizhevsky and Hinton, 2009).
The CIFAR-10/100 dataset is composed of generic images consisting of 10/100 classes, respectively.
To make a corrupted test dataset, we apply 15 types of common image corruptions (e.g., Gaussian noise, shot noise) to the test dataset.
We call the corrupted dataset CIFAR-10C/100C (Hendrycks and Dietterich, 2019).
We use the highest level (i.e., level-5) of image corruption for this experiment.
We use ResNet-50 as a backbone network.
For a fair comparison, we use the released trained model of Liu et al.
(2021) and the same hyperparameters whenever possible.
The number of nearby support examples N s is set to 1, the number of gradient steps per adaptation T is set to 1, the number of adaptation modules N e is set to 20, the number of support examples per each class M is set to 100, and the test batch size is set to 128.
We run experiments using four different random seeds.
More experimental results with other hyperparameter combinations are summarized in Appendix C.
4.2. E XPERIMENTAL RESULTS
The overall experimental results on CIFAR-10C/100C are summarized in Table 3.
We compare our method with the existing TTA methods and the online version of TTT++ (Liu et al., 2021), which is one of the state-of-the-art methods. TTT++ is one of the test-time training methods, which finetunes the feature extractor using the instance discrimination task along with matching the feature statistics of training and test time.
In Table 3, we observe that the TTA methods which use the frozen feature extractor such as T3A and TAST show slight performance changes compared to existing TTA methods that fine-tune the feature extractor.
On the other hand, TAST-BN outperforms all the TTA methods and TTT++ and it achieves better performance by 1 . 25% for CIFAR-10C and 4 .
56% for
CIFAR-100C on average, respectively.
Refer to Appendix D for the detailed experimental results on 15 types of image corruptions.
R ELATED WORKS
Test-time training methods Test-time training methods fine-tune trained classifiers using the selfsupervised learning task used at training time.
Sun et al.
(2020) uses a rotation prediction task (Feng et al., 2019), which predicts the rotation angle of the rotated images.
Liu et al.
(2021) use an instance discrimination task (Chen et al., 2020).
However, TTA methods, our focus in this paper, have no access to any information related to the training procedure.
We empirically demonstrated that our method outperforms the existing test-time training methods on the image corruption benchmark even without the knowledge about the self-supervised learning task.
Source-free domain adaptation methods Source-Free Domain Adaptation (SFDA) methods (Liang et al., 2020; Ishii and Sugiyama, 2021; Yeh et al., 2021; Eastwood et al., 2022) aim to adapt trained classifiers to unseen test domains without training dataset.
SFDA methods mainly focus on the setting that they can access the whole unlabeled test data, whereas TTA methods can access the online unlabeled test data only.
Recently, several SFDA methods using nearest neighbor information (Tang et al., 2021; Yang et al., 2021) have achieved good performances in domain adaptation benchmarks.
Especially, NRC (Yang et al., 2021) is built on the similar intuition that a test data and its nearest neighbors share the same label under domain shift.
However, unlike NRC, TAST utilizes adaptation module structures and prototype-based classification.
C ONCLUSION
We propose a simple and efficient test-time adaptation method, named TAST, that defines new pseudo-label distributions for test data using the nearest neighbor information and improves the classification performance by using prototype-based classification in the embedding space of adaptation modules trained with the pseudo-label distributions.
We demonstrate that our method outperforms the state-of-the-art test-time adaptation methods on domain generalization and image corruption benchmarks.
To the best of our knowledge, our work is the first work that utilizes adaptation modules which are randomly initialized at the beginning of test time and trained only a few times during test time.
We expect that utilization of adaptation module can be combined with the other methods in source-free domain adaptation or test-time training.
A I MPLEMENTATION D ETAILS A. I MPLEMENTATION DETAILS ON DOMAIN GENERALIZATION BENCHMARKS
We follow the dataset splits and the hyperparameter selection method used in T3A. We split each dataset of training domains into training and validation sets.
The training and validation sets are used for network training and hyperparameter selection, respectively.
Specifically, we split each dataset into 80% and 20% and use the smaller set as the validation set.
We choose the hyperparameters that maximize the validation accuracy of the adapted classifier.
This hyperparameter selection method is called the training-domain validation.
We train backbone networks using four different learning algorithms: ERM, CORAL, MMD, and Mixup.
ERM is explained in Section 2 of the manuscript;
CORAL aims to obtain domain-invariant representations by aligning covariance matrices of training data and test data; MMD tries to match the training and test data distributions using the MMD measure; Mixup trains classifiers using mixed images/features and mixed labels created by linear interpolation of examples from the training domains.
We run experiments using four different random seeds: 0, 1, 2, and 3.
All the hyperparameters for training and test-time adaptation are taken from T3A and DomainBed.
We train the network with Adam optimizer with default hyperparameters introduced in DomainBed, e.g., a learning rate of 0.00005, a weight decay of 0, a dropout rate of 0, and a batch size of 32.
In addition to the hyperparameters for test-time adaptation described in Section 4.1.1 of the manuscript, there is one more hyperparameter β for the baseline methods.
The learning rate for test-time adaptation is obtained by multiplying β to the learning rate used in training time.
We set the confidence threshold for PL and PLClf to 0.9.
The possible values for β are set to 0.1, 1.0, and 10.0.
For
TAST-BN, we restrict the size of the whole support set to 150 due to effective memory usage and reduced runtime since the test data and the support examples are fed into the classifier for every test batch.
A. I MPLEMENTATION DETAILS ON IMAGE CORRUPTION BENCHMARKS
We use the same hyperparameters introduced in TTT++.
We train ResNet-50 for 1000 epochs using the classification and instance discrimination tasks jointly.
The weight on the instance discrimination task for balancing the two tasks is set to 0.1.
For the instance discrimination task, we use the same data augmentation schemes of TTT++, e.g., RandomResizeCrop, RandomHorizontalCrop,
HorizontalFlip, ColorJitter, RandomGrayscale, and Normalization.
We set the batch size for training the networks to 256.
At test time, PL, SHOT, and TTT++ use SGD optimizer with a learning rate of 0.001 and a momentum of 0.9.
On the other hand, Tent, TAST, and TAST-BN use Adam optimizer with a learning rate of 0.001.
We set the batch size to 128 during the test time due to effective memory usage.
We run experiments using four different random seeds: 0, 1, 2, and 3.
We set the confidence threshold for PL and PLClf to 0.9.
For PL, we adjust only the BN layers in the trained model as in Tent.
For TAST-BN, we restrict the size of the whole support set to 200 .
However, even in CIFAR-100C experiments, we can store only two support examples per class if the support set size is fixed at 200.
Thus, we do not restrict the size of support set for TAST-BN on CIFAR-100C.
A. R UNTIME COMPARISON
We conduct our experiments on TITAN XP.
We report the average runtime spent to adapt classifiers that use ResNet-18 as a backbone network in Table 4.
We note that TAST, which updates the support set and the adaptation modules, requires only 1/3 to 1/4 running time compared to the methods that update the entire feature extractors, e.g. SHOT or SHOTIM.
On the other hand, TAST-BN, which updates the support set as well as the BN layer, requires more running time (about 2x) compared to SHOT or SHOTIM. The overhead is not significant though due to the online setting.
A. D ETAILS ABOUT ADAPTATION MODULES
We use BatchEnsemble (BE) for the adaptation modules of our method. BE is a simple and efficient ensemble method that greatly reduces the computational cost by weight-sharing. Each ensemble member of BE is composed of two layers with a shared weight and rank-one factors. Specifically, the weight matrix of j -th ensemble member is W ◦ r j s T
j where W is a shared weight and r j s T j is [Table]
the rank-one factor of j -th ensemble member. Although the existing deep ensemble (DE) methods do not share any weights, all ensemble members share W [Table] and thus BE reduces the number of parameters compared to DE.
Moreover, unlike DE, only the last layer of all ensemble members of BE are different, and thus it can be easily vectorized and trained simultaneously. Therefore, BE greatly reduces the computation cost. The adaptation module structure is used in many fields such as self-supervised learning (which is called “projection head”).
Although the existing methods mainly focus on training time, TAST focuses on test time. For example, SimCLR (Chen et al., 2020) adds a projection head on the top of a feature extractor at the beginning of training time and trains the feature extractor and the projection head with an instance discrimination loss. After the training time, for downstream tasks, SimCLR uses feature extractor outputs rather than projection head ones. However, TAST adds adaptation modules at the beginning of test time and trains the modules with the nearest neighbor-based pseudolabel distribution.
To predict the label of test data, we use the averaged predicted class distribution from the adaptation modules.
A. E NSEMBLE SCHEME IN TEST - TIME ADAPTATION
BACS (Zhou and Levine, 2021) incorporates a Bayesian inference framework into the TTA setting. BACS learns an approximate posterior for model parameter during training time, and adapts the trained model with a regularization term induced by the approximated posterior during test time. The main difference between ensemble schemes of TAST and BACS is the timing when the ensemble of models is built. BACS constructs the ensemble of predictive models at the beginning of training time and trains the models independently during training time.
Also, the ensemble members are individually adapted during test time. However, TAST focuses on building an ensemble of adaptation modules at the beginning of test time.
B P SEUDOCODE FOR TAST-BN
We present the pseudocode for TAST-BN in Algorithm 2.
TAST-BN fine-tunes the BN layers in the feature extractor instead of adaptation modules.
Since the embedding space of the feature extractor steadily changes, the support set stores the test data itself instead of the feature representations.
Formally, a support set S t = { S 1 t , S 2 t , . . .
, S Kt } is a set of test samples until time t .
The support set is initialized as an empty set.
At the time t , the support set is updated as
[Equation]
where p k is the likelihood the classifier assigns x t to the class k .
Using the support set, we retrieve
N s nearby support examples of x in the embedding space of f θ , i.e.,
[Equation]
(11) where β x is the distance between x and the N s -th nearest neighbor of x from S in the embedding space of f θ .
Then, we generate a pseudo label distribution for the test data and fine-tune the BN Require: Feature extractor f θ , test batch B , support set S , number of gradient steps per adaptation T , number of support examples per each class M , number of nearby support examples N s , learning rate α Ensure: Predictions ˆ y x for all x ∈ B
Update the support set S with eq.
(10) in Section B
Retrieve the nearest neighbors N ( x ; S ) for all x ∈ B with eq.
{+(11) in Section B for t = 
+}
: T do
Compute prototypes { µ k } Kk =1 using the support set in the embedding space of f θ for z ∈ N ( x ; S ) do p
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
end for for x ∈ B do
[Table]
end for θ
[Table]
end for for x ∈ B do p
[Table]
[Table]
end for layers to match the nearest neighbor-based pseudo label and a prototype-based class distributions for the test data with the same procedure described in Section 3 of the manuscript.
C A DDITIONAL E XPERIMENTS C
E XPERIMENTAL RESULTS USING CLASSIFIERS TRAINED BY DIFFERENT LEARNING ALGORITHMS
In Table 5, we show the results of test-time adaptation methods using classifiers trained by three different learning algorithms, namely CORAL, MMD, and Mixup.
TAST consistently enhances the performance of the trained classifiers on the benchmarks by 1 .
73% , 1 . 81% , and 2 .
30% on average using the classifiers trained by CORAL, MMD, and Mixup, respectively.
We find that TAST has a minor performance gain compared to the results in Table 1 of manuscript, whereas it surpasses T3A on most of benchmarks.
Compared to T3A, TAST shows better performance on the benchmarks by 0 .
14% , and 0 .
40% on average with the classifiers trained by CORAL, MMD, and Mixup, respectively.
Refer to Section 4 in Appendix D for the experimental results of the other baseline methods.
C. F INE - TUNING BOTH ADAPTATION MODULES AND BN LAYERS SIMULTANEOUSLY
We consider a method, named TAST-both, that fine-tunes both the attached adaptation modules and the BN layers in the feature extractor simultaneously.
[-(11) in Section B for t = 
-]
Table 6 reports the experimental results using classifiers learned by ERM on domain generalization benchmarks.
We use ResNet-18 as a backbone network.
As shown in Table 6, TAST-both shows worse performance than TAST-BN and
TAST.
We conjecture that the random initialization of adaptation modules and the changes in feature representation due to BN layer training negatively affect the learning of the other layers.
C. E XPERIMENTAL RESULTS USING DIFFERENT HYPERPARAMETERS ON CIFAR-10C
In Table 4 of the manuscript, we report the experimental results when N s and M are set to 1 andon the CIFAR-10C, respectively.
In Table 7, we summarize the experimental results using different combinations of N s and M on the CIFAR-10C. There are two observations in Table 7: (1) T3A has shown the best performances when M is set to 100; and (2) TAST and TAST-BN perform better with smaller N s .
[Table] [Table]
C. S ENSITIVITY ANALYSIS ON HYPERPARAMETERS
We follow the hyperparameter selection method used in T3A. We split the dataset of training domains into training and validation sets.
The validation set is used to select hyperparameters that maximize the validation accuracy of the adapted classifier.
On the other hand, for the image corruption benchmark, we use manually determined hyperparameters as in Tent.
Thus, we summarized experimental results on other combinations of hyperparameters in Table 8-11.
Additionally, we investigate the sensitivity of two hyperparameters which are set manually throughout all experiments, the softmax temperature τ and the output dimension of adaptation modules d ϕ .
We set τ and d ϕ to 0.1 and d z / 4 , where d z is the output dimension of the feature extractor.
In Table 8-11, we report the average accuracy of the adapted classifier by TAST with the different combinations of τ and d ϕ .
In the experiments, we use ResNet-18 as a backbone network trained by
ERM on PACS, which is one of the domain generalization benchmarks.
We experimentally show that the performance of TAST is robust to changes in τ and d ϕ .
We especially think that the classification performance of TAST is not significantly affected by changes in τ because τ affects both the prototype-based predicted class distribution of test data and the new pseudo-label distribution using nearest neighbor information and then we train the adaptation modules with the cross-entropy [Table] loss affected by τ only a few times per each test batch during test time.
Moreover, we can observe a similar classification performance regardless of the dimension of adaptation modules similar to
Chen et al.
[Table]
[Table]
[Table]
We used the test batch size as in T3A and Tent for domain generalization and image corruption benchmarks, respectively, as described in Appendix A and Section 4 of the manuscript.
We summarize experimental results using different test batch size.
We conduct experiments using classifiers, which have ResNet-18 backbone networks, learned by ERM on PACS.
As shown in Table 12, we can find that Tent and PL show reduced performance in experiments using smaller test batch size, but T3A, TAST, and TAST-BN are robust to changes in test batch size. [Table] [Table]
[Table]
D F ULL R ESULTS
[Table]
ERM for Table 1 of the manuscript on PACS.
We use ResNet-18 as a backbone network.
[Table]
We use ResNet-18 as a backbone network.
[Table] [Table]
[Table]
ERM for Table 1 of the manuscript on PACS.
We use ResNet-50 as a backbone network.
[Table]
We use ResNet-50 as a backbone network.
[Table]
[Table]
[Table]
CORAL for Table 21 on VLCS.
We use ResNetas a backbone network
[Table]
[Figure] [Table]
[Table]
TerraIncognita.
We use ResNet-18 as a backbone network
[Table] [Table]
[Table] [Table]
TerraIncognita.
We use
ResNet-50 as a backbone network.
[Table]
, OfficeHome, and TerraIncognita. We use ResNetas a backbone network [Table] as a backbone network
[Table] [Table]
30 on OfficeHome.
We use ResNet-18 as a backbone network.
[Table]
[Figure]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table] [Table] [Table]
[Table]
We use ResNet-18 as a backbone network
[Table]
We use ResNet-18 as a backbone network.
[Table] [Table] [Table]