Test-Time Classifier Adjustment Module for Model-Agnostic Domain Generalization
Authors: Yusuke Iwasawa
Yutaka Matsuo
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: domain generalization
test time adaptation
prototypical classifier

Abstract: This paper presents a new algorithm for domain generalization (DG), \textit{test-time template adjuster (T3A)}, aiming to robustify a model to unknown distribution shift. Unlike existing methods that focus on \textit{training phase}, our method focuses \textit{test phase}, i.e., correcting its prediction by itself during test time. Specifically, T3A adjusts a trained linear classifier (the last layer of deep neural networks) with the following procedure:  (1) compute a pseudo-prototype representation for each class using online unlabeled data augmented by the base classifier trained in the source domains, (2) and then classify each sample based on its distance to the pseudo-prototypes. T3A is back-propagation-free and modifies only the linear layer; therefore, the increase in computational cost during inference is negligible and avoids the catastrophic failure might caused by stochastic optimization. Despite its simplicity, T3A can leverage knowledge about the target domain by using off-the-shelf test-time data and improve performance. We tested our method on four domain generalization benchmarks, namely PACS, VLCS, OfficeHome, and TerraIncognita, along with various backbone networks including ResNet18, ResNet50, Big Transfer (BiT), Vision Transformers (ViT), and MLP-Mixer. The results show T3A stably improves performance on unseen domains across choices of backbone networks, and outperforms existing domain generalization methods. 

Test-Time Classifier Adjustment Module for Model-Agnostic Domain Generalization
Yusuke Iwasawa The University of Tokyo
Yutaka Matsuo The University of Tokyo
Iwasawa@weblab. u-tokyo.... ac. ip
matsuo@weblab. wwwooky......oo..oooooooooooo... ac. ip
Abstract
This paper presents new algorithm for domain generalization (DG), test-time template adjuster (13A), aiming to robustify a model to unknown distribution shift. Unlike existing methods that focus on training phase, our method focuses test phase, i.c., correcting its prediction by itself during test time. Specifically, T3A adjusts trained linear classifier (the last layer of deep neural networks) with the following procedure: (1) compute pseudo-prototype representation for each class using online unlabeled data augmented by the base classifier trained in the source domains, (2) and then classify each sample based on its distance to the pseudo- prototypes. T3A is pacckpropagation-to frre and modifies only the linear layer; therefore, the increase in computational cost during inference is negligible and avoids the catastrophic failure might caused by stochastic optimization. Despite its simplicity, T3A can leverage knowledge about the target domain by using off-the-shelf test-time data and improve performance. We tested our method four domain generalization benchmarks, namely PACS, VLCS, OfficeHome, and Terralncognita, along with various backbone networks including ResNet18, ResNet50, Big Transfer (BiT), Vision Transformers (ViT), and MLP-Mixer. The results show T3A stably improves performance on unseen domains across choices of backbone networks, and outperforms existing domain generalization methods.
Introduction
Deep neural networks often fail to generalize -fd-striiitittto samples. Accuracy suffers when the model performs under conditions different to those of training, such as variations in light [8], weather [51], object poses [2]. textures [16], object backgrounds [5]. Nevertheless, the model may be deployed to different conditions practical situations; thus, some countermeasures are needed.
Over the past decade, various studies have focused on training a generalizable model to unseen domains given dataset consisting of several source domains. This setting is usually denoted as domain generalization (DG) [6, 61]. Domain generalization operates under the assumption that one can improve robustness to domain shift by incorporating the structure common to multiple domains. For example. domain-invarian feature learning constrains the representation to be invariant to domain shifts [14, 45, 29]. Other methods use meta-learning [19] to learn how to regularize the model to improve the robustness [28, 4, 31]. However, despite significant work on this front, machine learning systems are still vulnerable to domain shifts even after using the above methods during training. Notably, recent large-scale benchmarks [17] show that many approaches do not provide significant improvement compared to simple supervised learning, i.e., empirical risk minimization (ERM), with proper and practical experimental setup. suggests that the setup in its current state may be too difficult, and a different approach might be needed.
This paper proposes method of using additional off-the-shelf data in the DG setup, the unsupervised data available at the test-time. Since no data about the target domain is available during training in
35th Conference on Neural Information Processing Systems (NcurIPS 2021), virtual.

---

DG setup, the existing domain generalization algorithms focus on how to use labeled data from multiple-souree domains. However, test-time the model always has access to test data from the target domain. Although the available data is constrained to be (1) unlabeled and (2) only available online (models can not know all test cases in advance), this data provides clue about the target distribution that is not available during training. It is natural to ask the question: How can we use the off-the-shelf unlabeled data available at test-time to increase performance on the target domain?
It is worth emphasizing that our setting is different from the transductive setting [49, 22, 57] where all test cases are known in advance, even though we use test data for adjustment. When testing, the model is usually deployed in some environment, and must work well on various samples that will appear continuously. Similarly, the deployed model usually needs to make correct predictions at that moment; there is no point in going back in time and conrecting the predictions. Therefore, it is desirable that adjustment and inference be performed at the same time, not offline after large amount of data has been accumulated. Looking beyond domain generalization, some recent studies suggest optimizing the model during test time using objective function defined only by unsupervised data (e.g, prediction entropy) [34, 52]. However, updating parameters using stochastic gradient descent (SGD) increases computational costs and harm inference throughput. In addition, data available at test time is limited, and stochastic optimization can lead to catastrophic failure.
To this end, we present test-time templates adjuster (T3A), which adjusts the linear elassifier (the last layer of deep neural networks) at test-time. T3A adjusts the weights of the linear classifier for each class using online unlabeled data and the classifier trained in the source domains, (2) and then classify each sample based on its distance to the geeudo-prototyen boundary avoid the high-data density region on the target domain and reduce the ambiguity (entropy) of predictions, which is known to be connected to classification error [52]. Since T3A does not alter the training phase, it can be used together with existing DG algorithms. Moreover, it can be used together with any classification model since it only adjusts the linear classifier on top of the representations. Some readers may wonder how effective it is to modify only the linear classifier while freezing the representation itself. Later in this paper (Section 3.2), we empirically demonstrate that this modification is indeed beneficial.
as the following optimization- free procedure: (1) create pseudo-proton
This procedure makes the adjusted decision
We evaluate our method on multiple standard domain generalization benchmarks, namely VLCC [12], PACS [27], OfficeHome [50], and Terralncognita [5]. We compare our method with (1) various DG algorithms reported [17] and (2) Tent [52] that minimizes the prediction entropy at test time using SGD. With the standard ResNet50 backbone [18]. T3A improves ERM by 1.5 points
average accuracy over four dataset, and outperforms most existing DG algorithms. Furthermore, we evaluated our method with 10 different backbone networks, including residual networks (resnet18 88 and resnet50), big transfer (BiT-M-R50x3, BiT-M-RIO1X3 and BiT-M-RI 52x2 [24]), vision transformers (ViT-B16 16, ViT-L16, Hybrid ViT [91, DeiT [48],, and MLP-Mixer (Mixer- L16) [47]. The results show that T3A gives a statistically significant performance gain against ERM on all backbone networks.
2
Preliminary and Related Work
2.1 Domain Generalization
Problem setup Following [6], we assume multiple datasets Dd = [(x,,, 1 collected from several different domains d E
). The dataset Dd from domain d contains identically and independently distributed samples characterized by some probability distribution Pd(x, Y), where X and Y are random variables of input and target, respectively. Then, our goal is to develop a predictor f(X) that performs well some unseen test domain, which is characterized by a different probability distribution P(X,Y) * (=(X,X Y) for all d E {1,... der Note that one can not assume the target distribution during training. e.g., no data about the target distribution is available at the time. Therefore, the predictor is usually trained on datasets from several source domains. For example, the predictor can be trained by minimizing the empirical risk:
arg min 3
((f(x))) M.).
where Φ the set of the parameters of the function, and l a loss function measuring prediction error. In the rest of this paper, optimizing the predictor with eq. 1 is called ERM. In a real application,
2

---

the model will be deployed after training and expected to classify the data in an online manner. For benchmarking the algorithm, given a dataset containing na domains, we usually use the leave- one-domain-oue procedure, which uses a single domain as a test domain and the others as training domains. The procedure is repeated na times, changing the test domain every time.
Algorithms A central branch of DG algorithm is domain-invarian feature learning, which explicitly reduces the domain gaps on space of latent representations. For example, [14] proposed domain- adversarial networks (DANN) which measure the domain gaps via an external domain classifier. CORAL [45] align the second-order statistics of representations among different domains. [29] uses maximum mean discrepancy (MMD) to measure the domain gap. Many extension have been proposed [33, 1, 21], but they are all the same in that they enhance domain invariance. Another branch are meta-learning-basee based methods [28, 4, 31], which divide the available domains into meta-train meta-train-domains and meta-test-domain and regulate the model trained in meta-train-domant meta-test-domain. Invariant risks minimization [3] regularizes ERM with gradient normalization penalty over a dummy classifier. Several studies propose to augment the data using mixup [59] between two source domains, which implicitly enhances invariance to domain shifts [56, 58, 53].
to be useful for the
Key differences As briefly mentioned above, existing domain generalization algorithms focus on the training phose; how to regularize the predictor using the knowledge from multiple source domains. Our work focuses on the test phase; how to adjust the model using online and unlabeled data, which can characterize the target distribution. Note that proposed method works fully online; It does not require access to offline unlabeled data, and therefore can be compared fairly with existing DG methods
2.2 Other Related Work
Unsupervised domain adaptation. Our work is related to unsupervised domain adaptation (UDA) [37, 38, 55] as both methods aim to adapt a model given unsupervised data. However, our work primarily differs from UDA in that UDA focuses on adapting during training, while we focus on adapting during testing. In other words, UDA assumes we can access labeled data from the source domain and (unlabeled) data from the target domain the same time, which is not always possible.
Source-free domain adaptation. Among them, recent "source-free" setups are particularly similar to our setting [30, 25, 34]. In these setups, source data is not needed during the adaptation phase, and the model is adapted using the unlabeled data solely from the target domain. However, this adaptation usually made in an offline manner, i.c., these source-free methods optimize offline with multiple losses for multiple epochs. Our method adjusts the classifier in an online-manner, and therefore is suitable for domain generalization setup where the trained model is assumed to be deployed.
Test-time adaptation. Regarding the problem setup, our work is most closely related to test- time adaptation [52] test-time training [46]. Notably, [52] proposes fully test-time adaptation, which modulates the BN parameter by minimizing the prediction entropy using stochastic gradient descent. The concept of test-time adaptation is very similar to our work and can be used in domain generalization, however, it has not been fully investigated under the domain generalization setup. Moreover, recent architectures do not employ batch normalization either on pre-training (mainly to avoid the large memory usage required by BN) or fine-tuning phase (for improving performance). Besides, minimizing the prediction entropy using SGD could lead to trivial solutions, such as being biased to predict only particular class. In comparison, (1) our method can be used together with any classification models since it only adjusts the linear classifier on the top of the representations, (2) our method alleviates catastrophic forgetting due to not using SGD during test-time.
Prototypical networks. Before deep learning become popular, the prototype-based classifier is well-investigated in the context of semi-supervised learning [10]. continual learning [35, 39], and few-shot learning [44]. Our work most similar to prototypical networks [44], which combine prototypical classifier and deep neural networks as with our method. However, the use-cases mentioned above of prototypical networks assume access to a few labeled data from the same domain, which differs in our case. To handle the difference, we combine prototypical networks with pseudo- labeling techniques, which are often used in domain transfer literature [42]. Besides, connection to entropy minimization is new perspective introduced in this paper.
3

---

Algorithm Algorithm T3A for prediction.
Input:
Feature extractor fe. the batch of imput B. and support sets S available this point. Output: Prediction for all æ E IB, where æ ~ P(X). # Stepl. Adjust the template for each class using the IP.
for о E IB do
I = arg max qw (Y (x)))(())) (eq. 2)
g = s U
Io(m) [o(())||
} for y² (eq. 3)
end for
Filter support sets with eq. 6
# Step2. Predict based on the distance between the adjusted template. return arg max 7(Y = yxkle((a)) for all (eq. 4)
3 Proposal: Optimization-Free Test-Time Classifier Adjustment Module
We propose to replace the output layer of the predictor trained on source-domains (i.e.. linear classifier) sveudo-prototypicu classifier, whose prototype features are adjusted during test time while fixing the features already trained on the source domains. We call our method T3A, for Test-Time Templates Adjuster. We first explain the detailed algorithm (Section I), and explain how and why it works (Section 3.2). Algorithm 1 outlines the procedure.
3.1 Algorithm
We assume the predictor is deep neural networks (DNN) obtained by some learning algorithm (e.g.. ERM, DANN, CORAL, etc.) using data from source domains. For convenience, the entire DNN is divided into a linear classifier qu for the last layer and a feature extractor for the rest, where and are the parameters of neural networks. In usual domain generalization setup. ʃo and qw are used to predict data from the test domain. For new data æ. the prediction is given by taking the argmax over the following approximated probability distribution:
cxp(2 - w*)
arg max (Y = [olff(z)) Σ, expp(= wi)'
(2)
where = = fo(x), w E Re is the k-th element of the weights matrix in w, and Zdim is the dimension of z, which depends on the feature extractor fe. In this prediction, w works as the template of representation for the class k, and prediction is done by measuring the distance (dot product) between the template and the representation of the input data. Since this template was trained in the source domain, there is guarantee that it will be good template in the target domain.
T3A adjusts the templates during test-time. Assume we have (batch of) test-data æ at time t drawn from target distribution P(X)= * pd(X) for all d E {1.... d.r). As each prototype should be related to some class, we first augment the input data a via pseudo label ÿ, which is obtained via eq. 2. Then, we update support set S as follows:
Sh gk
(a(
if 0 = y* else,
(3)
where ||a|| represents the L.2 norm of the vector a and S = { ||اا|| }. If the input data contains multiple samples at the same time (e.g., a batch of data), the above procedure is repeated for each sample in the batch. Then, prediction is done by taking the argmax over the following adjusted probability distribution:
exp(2 c²)) exp(x ci)
arg max Yo Volfo(x)) Uk
(4)
where ch are the centroids of sk:
(5)
Sk
ZeSk

---

Eitopy Rein
Extapy Auan
0.0
Seen Uts4
EAM
-T3A
-Tere.c
0.4
VL25
16C3
OliaPini lara
VLSS
196.5
Offeateme
Tamu
V253
[ACS
Offcatomma
Noma
Dolaest
Datases
Dotasat
(a) Prediction entropy
(b) ERM vs. T3A vs. Tent
(c) Transfcrability
Figure I: Pre-experimental results. (a) Comparing the entropy of predictions (with ResNet50) the source domains and the target domain. The domain shift increases entropy. (b) T3A effectively
reduces entropy. (c) Transferability of each components of the model trained by ERM in each dataset.
Note that, this procedure will be repeated each time new data arrives, without discarding the support sets, which may make infeasible to retain all past data in eq. 3. Also, some pseudo-labels assigned to the wrong class, and using this data is not desirable as adds noise to the templates and may deteriorate performance. To avoid this issue, we use the prediction entropy Hw =) = Ex f (Y- 1=) |z) log (Y = y*1²) to filter unreliable pseudo-labeled data. Specifically, before making prediction using eq. 4, only a part of the support set restored as follows:
s {
SN H H [2) S a²}.
(6)
where a is the M-th largest entropy of the support set S (M is hyperparameter).
3.2 Remarks
Remark 1: T3A implicitly reduces prediction entropy. As prior works suggest [52], the predic- tion entropy is often related to an error, as more confident predictions tend to be more correct. Figure 1-a shows that the prediction entropy also characterizes the difficulty in DG setup; entropy in the unseen domain tends to be greater than entropy in the seen domains. To be more specific, we first trained ResNet50 on the source domain by ERM. The training was done leave-one-domain-o manner, and we conducted three experiments with a different seed each time. We used four standard datasets in domain generalization (VLCCS [12], PACS [27], OfficeHome [50], and Terralncognita [5]). We used the implementation of Doma BemaiinBed [17], and used the default hyper-parameter for pre-training on source- cource-domains and fine-tuning on target domain: namely, we use Adam [23] with learning rate of 5e-5 for optimization and use batch size of 32 with no dropout or weight decay.
With this in mind, existing studies have modified the model parameters to explicitly reduce entropy. Although the proposed method does not explicitly reduce entropy, it has the effect of implicitly reducing it. This because the proposed method uses a template updated with samples from the target distribution, which provides a decision boundary that avoids the dense parts of the target distribution. Figure 1-b compares the prediction entropy on the target domain among (a) ERM without test-time modulation, (b) T3A, (c) Tent-C, which updates the classifier to minimize entropy. The results show that T3A can effectively reduce entropy without using online optimization. Note that changing hyper-parameter (such as learning rate) might change the results for Tent-C, but it may corrupt the entire classifier. See Section for the hyperparameter selection.
Remark 2: T3A is computationally light. Unlike Tent, our method does not use SGD. Besides, the representations are fixed, and is not necessary to repeat the forward propagation of the feature extractor. The only computational overhead is the cost of one forward propagation of the last linear layer, which is usually negligible compared to the forward and back propagation of feature extractors. Since it is not desirable to reduce throughput when considering online prediction, the proposed method is suitable in this respect as well.
Remark 3: Adjusting the linear classifier can significantly improve performance. Some readers may wonder how effective to modify only the linear classifier while freezing the representation. To answer this question, compare the DG performance (None) before fine-tuning,
5

---

(head) after fine-tuning only qw. (body) after thne-tuning only fe. and (all) after fine-tuning the entire network (Figure 1-c). Each block corresponds to different dataset, and each color represents a different fine-tuning strategy. The results show that fine-tuning only the classifier often significantly improves performance. For example, in VICCS, the average performance score jumps from 72.5 to 82.9, which is close to the 84.7 obtained when the entire network is fine-tuned. The performance gain differs for each dataset, but the tendency is generally the same. In addition, this tendency was the same for other backbone networks including BiT, ViT, and Mixer (see Appendix B). These results indicate that adjusting only the linear classifier can significantly improve performance in various configurations. Note that the number of parameters of the linear layer is much smaller than those of the feature extractor in standard network architectures.
4 Experiment
We evaluate T3A on four standard domain generalization benchmarks, namely VLCS [12]. PACS [27], OfficeHome [50], and Terralncognita [5]. Our implementation uses the Doma i nBed library [17]'. We modify Doma i BBed (1) to use various backbone networks using the mm library [54]], and (2) to implement test-time adaptation algorithms (ours and Tent). For Tent, we used the original implementation'. We run our experiments mainly on cloud V100x4 or A100x8 instances, depending on the memory usage of the backbone networks. See Appendix A for more information, including licensing information and total amount of compute.
Datasets. VLCS [12] comprises four photographic datasets d E [Caltech101[13] LabelMe[40], SUN09[77, VOC2007[11]}, containing 10, 729 examples of 5 classes. PCAS [27] comprises four domains d E (art, cartoons, photos, sketches), containing 9, 991 examples of classes. OfficeHome [50] includes domains d E (art, clipart. product, real), containing 15, 588 examples and 65 classes. TerraIncognita [5] includes photo of wild animals taken by camera at different locations. Following [17]. we used datasets of d E {L100, L38, L43, L43, containing 24, 788 examples and 10 classes.
Backbone networks. For the main experiments, we use residual networks with 50 layers (ResNet50). which was the default setting of the prior studies. In addition, we tested our algo- rithms on 10 different pre-trained models: residual networks with different layers (ResNet18 and ResNet50), Big Transfer [24] with different layers (BiT-M-R50x3, BiT-M-R101X3 and BiT-M- R152x2), ). Vision Transformers [9] with variations (ViT-B16, ViT-L16, HViT, which uses ResNet50 as patch embedding of ViT, DeiT [48],, and MLP-Mixer (Mixer-L16) [47].
Baselines. We compare our method to domain generalization algorithms and test-time adaptation algorithms. For domain generalization algorithms,, we mainly compared with the results reported in [17]. These results include the following algorithms: Empirical Risk Minimization (ERM), Group Distributionally Robust Optimization (GroupDRO) [41], Inter-domain Mixup (Mixup) [56, 58, 53], Meta-Learning for Domain Generalization (MLDG) [28), DomainAdversant
Neural Networks (DANN) [15], Class- colasssconiiionall DANN (C-DANN) [32], Deep CORrelation ALignment (CORAL)
[45], Maximum Mean Discrepancy (MMD) [29]. Invariant Risk Minimization (IRM) [3], Adaptive
Risk Minimization (ARM) [60], Marginal Transfer Learning (MTL) [6] Style-Agnosic Networks
(SagNet) [36], and Representation Self Challenging (RSC) [20].
We also compared our method with existing test-time adaptation methods. Note that we can not simply use BN-based methods (including Tent [52], which is the most up-to-date method) on the DG setup because [17] omit the BN layer from pre-trained ResNet when fine-tuning source domains. Besides, several backbone networks evaluated in our paper do not contain BN from the beginning. Therefore, we first tested two slightly modified versions of Tent on standard DG setup (Table 1, Table 2, and Figure 2). Specifically, Tent-C modulates the entire classifier to reduce prediction entropy. Tent-BN adds one BN layer just before the linear classifier and then modulates BN's normalization and transformation parameters. We then compare T3A with other test-time adaptation methods (including SHOT [34), pseudo labeling (PI.) [26), Tent-Full [52], BN-Norm [43]) using 1 and ResNet50 without removing batch normalization layer (Table 3).
P hhups://githubbbc http://hhhw..coomm///ig/m///nnnp http:://gthubb.com http:///gghhbbooomoDuuuuaaaaaatt

---

Table 1: Domain generalization accuracy for datasets and algorithms. Bold type indicates
performance improvement from the base model, and indicates statistical significance in one-sided paired I-lest (** indicates p S 0.01, * indicates S 0.05).
Algorithm
VLCS
PACS
OfficeHome
Terra
Avg
ERM
77.5 + 0.4
85.5 + 0.2
66.5 + 0.3
46.1 1.8
69.0
IRM
83.5 + 0.8
64.3 2.2
47.6 0.8
68.5
GroupDRO
0.6
84.4 14.40.8
66.0 0.7
4
67.6
Mixup
84.6 0.6
6.3
4.8
69.5
MLDG
0.4
8.0
6.6
4.9
69.2
CORAL
0.6
8.3
6.3
1.0
70.3
MMD
0.
8.
6.13
4.6
67.7
DANN
0.4
8..6
6.69
4.5
68.7
CDANN
0.1
82
1.
4.6
67.9
MTL
0.4
0.56
6..4
4.2
68.5
SagNet
0.5
0.2
68.1
48.6
70,2
ARM
0.3
0.4
6.3
4.3
68.3
VREx
0.2
8.69
6.64
0.6
69.0
RSC
77.1 0.5
85.2
6.
46.6 1.0
68.6
ERM
77.7
0.9
6.34
16.5
68.6
+T3A (Ours)
0.2
85.3 0.6
68.3
47.0
70.1 TM
+Tent-BN
0.2
84.8 0.5
6..0
0.3
66.2
+Tent-C
7.4
82.3 + 1.2
6..7
45.5 0.4
67.6
CORAL
78.6
84.2 0.3
6.1
48.1
69.8
+T3A (Ours)
+
85.6 0.2
69.2
47.3 0.7
70.4"
+Tent-BN
71.4 0.7
85.6 0.2
69.2
46.5 16.5 0.5
68.2
+Tent-C
78.1 + 0.5
83.7 + 0.4
68.2
47.8 + 1.1
69.5
Hyperparameters and model selection. As [17] claimed, model selection is not trivial in DG and significantly affects performance. We used standard training-domain validation for selecting hyperparameters, which uses the subset of each training domain to choose model. As reported in [17], we split the data from each domain into 80% and 20% splits and use larger splits for training and smaller splits to select hyperparameters. Following [17], we conduct random search of 20 trials over a joint distribution of hyperparameters to train the base model (see Appendix A.4).
In addition, T3A has one hyperparameter M for deciding the number of supports to restore, and Tent has two primary hyperparameters: 3 for multiplying the base learning rate (used for the base model) and y for the number of iterations per adaptation. It is worth emphasizing that these parameters should be selected before the deployment, i.e., before accessing the test data. We simply selected these hyperparameters by the average accuracy in the training- domain validation data when using these adjustment modules. Specifically, we tested M (1,5, 20, 50. 100, N/A) for T3A, where N/A means restoring all samples, and combination of E {0.1, 1.0, 10.0) and {1,3] for Tent.
4.1 Results
Table summarizes the results when ResNet50 is used the backbone network. The first block (from ERM to RSC) the value taken from [17]. The lines labeled ERM and CORAL are the scores reproduced in environments. The proposed method and Tent are based on this reproduced model. Figure 2 shows the distribution of performance improvement by the proposed method for models trained with different hyperparameters (20 3 for each test environment). In addition, Table 2 show the DG accuracy with 10 different backbones. Note that this experiment is conducted only on the default hyperparameter of ERM. Every number we report is mean and standard error three repetitions with different weight initialization, and dataset splits.
T3A stably improves the performance of the base model. The second block of Table shows that T3A stably improves the performance of the ERM model. Specifically, the proposed method improves 2.3 points, 2.0 points, 1.9 points, and 0.5 points for each dataset respectively. The average

---

Table 2: Domain generalization accuracy with different backbone networks. T3A increases the performance agnostic to backbone networks. Note that, this experiments is conducted only on the default hyperparameters ERM. Bold type indicates performance improvement, and * indicates statistical significance in paired t-test (** indicates p < 0.01, indicates p & 0.05).
Models
VLCS
PACS
OfficeHome
Terra
Avg
resnet18
1 0.9
80.3
55.7 0.2
40.7 1 0.3
62.5
+T3A
76.5 1 0.9
81.7 1 0.1
57.0 0.4
41.6 1 0.5
64.2*
resnet50
75.5
83.9 0.2
64.4 0.2
45.4 + 1.2
67.3
+T3A
78.3 + 0.7
84.5 + 0.3
66.5 + 0.2
45,9 + 0.5
68.8*
BiT-M-R50x3
76.7 1: 0.1
84.4 + 1.2
69.2 +: 0.6
52.5 + 0.3
70.7
+T3A
19.7...3
85.4 0.9
71.7 + 0.6
52.2
72.3*
BiT-M-R101x3
75.0
84.0 + 0.7
67.7 + 0.5
47.8 + 0.8
68.6
+T3A
78.6 + 0.4
85.4 + 0.5
69.9 0.4
48.1 + 0.8
70.5
BiT-M-RR12x2
76.7
0.1
71.3 + 0.6
51.4 1.6 0.6
71.1
+T3A
79.1 0.4
86.4
73.2 0.5
50.9 0.7
72.4"
VIT-B16
+.3
0.1
78.4
41.8
71.3
+T3A
80.2 1.4 0.4
8.10
78.9 0.3
42.5 0.7
71.9"
VIT-L16
0.5
0.
7.1
42.7
70,9
+T3A
79.0 0.6
85.5 0.6
78.7 0.2
4
72.1
DeîT
79.3 0.4
0.3 0.5
50.0 1 0.2
73.4
+T3A
81.3 1 0.4
89.5 0.4
78.3 0.2
50.1 + 0.2
74.8*
HVTT
79.2 + 0.5
89.7 0.4
80.0 + 0.2
51.4 + 0.9
75.1
+T3A
81.0 + 0.1
90.4 0.5
80.5 + 0.2
52.3 + 1.0
76.1"
Mixer-L16
76.4 1 0.2
81.3 1 1.0
69.4 1: 1.6
37.1 1: 0.4
66.1
+T3A
80.3 + 0.3
83.0 0.8
72.3 1.8
37.5 + 0.8
68.3*
10.0%
5.0%
5.0%
5.0%
AAcC
AAcc
AAcc
-5.0%
-5.0%
-5.0%
-10.0%
-10.0%
-1000%
5
C
P
Test Env
Test Env
Test Env
(a) ERM VLCS
(b) ERM PACS
(c) CORAL PACS
Figure 2: Distribution of performance improvements by the proposed method for models trained with different hyperparameters (20 for each test environment). Dashed line in each violin plot represents the quartile of the distribution.
improvement from ERM is 1.5 points. For clarification, paired t-test performed using 48 paired data (4 datasets, 4 test domains, different seeds). As results, the difference statistically significant (p < 0.01). Note that, Tent-BN improves the performance in PACS and OfficeHome, but it is not stable may be due to the failure of the optimization. Tent-C never improve the performance.
In addition, Fig. 2 show that the third-quarter quantile of the improvement range is often more than 0, which means the proposed method also robust the hyperparameters the base model. Furthermore, Table shows that the proposed method can also improve the performance of more sophisticated backbone networks. For example, the proposed method improves performance by an average of 1.0 points for HVTT, which achieved the best performance of all backbones. The

---

improvement by the proposed method is statistically significant (p S 0.05 for ViT-LI6, p - 0.01 for other backbones) with the one-side paired t-test.
T3A outperforms most existing DG algorithms. For example, in VLCS, the proposed method achieves 80.0%. which is significantly better than the prior best-reported score, 78.8%. Similarly. it is 85.3% for PACS, 68.7 68.3% (best) for OfficeHome and 47.0% for Terralncognita, and 70.1% average (third best). Note that the scores we reproduce are much worse than the reported scores in PACS, which reduces the average performance. PACS, the improvement from ERM by the proposed method is 1.7 points, which is the most significant improvement from any reported value.
For more accurate comparison, we also examined whether the proposed method would improve the performance of CORAL, which had achieved the best performance in existing reports. As shown in the third block of Table 1, the proposed method can also improve the performance of CORAL. The average improvement is 0.6 points. Average performance is 70.4%, which is the best among all algorithms. Note that, similar to ERM, the reproduced score and the reported score are slightly different (and the score in PACS is particularly low).
T3A outperforms existing test-time adaptation methods. We further compare T3A with other test-time adaptation methods when backbone networks employ the BN layer so that we can make a fair comparison with the prior BN-based methods [43, 52). We used ResNet 8 w/BN and ResNer50 w/BN as backhone networks. Note that the results of ResNet Table I and Table do not use the BN layer since it is the default option in [17]. Therefore, the results below are not directly comparable to the DG method shown in Table and Table 2 of the current manuscript.
We tested the following six baselines in addition to T3A, Tent-BN, and Tent-C. (1) Tent-Full updates BN statistics and transformations, which is the same as the original proposal [52]. (2) BN-Norm update BN statistics but fixes transformations parameters [43]. (3) PL. (Pseudo Label) [26] updates entire networks by minimizing the cross- cross-entropy between prediction and the pseudo label. Following [52], we assign the pseudo label if the predictions are over a threshold (0.9 in our experiment). (4) PL-C updates the linear classifier by minimizing the above-mentioned pseudo-label loss. (5) SHOT [34] updates feature extractor to minimize entropy, diversity regularizer, and pseudo-label loss. While [34] originally proposed SHOT in the context of source-free domain adaptation (offline adaptation setup), the method itself can be transferred to our setup. Note that the original SHOT uses the label-smoothing when training on the source domain. However, we focus on the adaptation method, and therefore the source model is the same as the other baselines for the fair comparison. (6) SHOT-IM [34] updates the feature extractor to minimize entropy and the diversity regularizer.
In summary, (1) all baseline except BN-Norm and T3A use stochastic optimization during test-time, which is not desirable since may cause catastrophic failure and must increase computational costs. (2) Felt-Full and BN-Norm are powerful yet constrained to be applicable only if the architecture uses BN.
Table 3 compares results under the training-domain model selection as with Table I and Table 2. For clarification, we also compare performance under the oracle model selection (28 in Appendix C), where one can use the validation set on the target domain (20% of all data as described above). We can make the following observations.
(1) The proposed method still outperformed all baselines in both backbone networks. Among baselines, only Tent-Ful Full and PL-C perform better than None (w/o adaptation) on average. (2) When we select the saprr-prrameterss with test-domain validation set, Tent-Full gives comparable performances with the proposed method. In addition, compared to T3A and PL--C, the T3A performs better under both model selection strategies. These results clarify the difficulty of model selection in optimization-basee methods and the merit of the proposed optimization-fie free approach in this setup. (3) Updating feature extractor (or the large portion of the parameters) does not work well in general, while it is common in SFDA (offline) setup. The results suggest that we need different treatments on online and offline setup.
5 Discussion and Conclusion
This paper presents new domain generalization algorithm, T3A, which adjusts its predictions during test-time by itself. We show that T3A reduces prediction entropy (Fig. 1-c), and more importantly, generalization error on unseen domain (Table 1). T3A can adapt different domain generalization

---

Table Comparison of our method and existing test-time adaptation methods. Unlike Table 1 and Table 2, we used ResNet18 1 and ResNet50 without removing batch normalization layer as backbone networks. with Table 2, this experiments is conducted only on the default hyperparameters of ERM. Bold type indicates performance improvement, and indicates statistical significance in paired 1-test (* indicates p < 0.05).
Models
VLCS
PACS
OfficeHome
Terra
Avg
resnet 8 w/ BN
73.0 0.6
79.5 + 0.4
61.8 + 0.3
41.7 + 0.9
64.0
SHOT-IM
0.3
82.1 + 0.3
62.5 0.3
32.8 0.4
59.8
SHOT
0.3
82.3 0.2
62.8 1 0.2
32.7
59.9
PL
6.6
1.0
56.3
35.4 1.7
57.9
PL-C
1.3
7..9
0.3
43.1
63.9
Tent-Full
83.9 0.3
6.27
0.3
64.0
BN-Norm
1.0
82.7 0.1
0.1
3.24
62.9
Tent-C
74.6 1.5
6..5
40.9
61.8
Tent-BN
64.7 0.7
81.1 0.2
0.3
3.
61.2
T3A (Ours)
74.5 + 0.9
81.4 + 0.2
63.2 0.4
39.5 - 0.3
64.6*
resnet50 w/ BN
74.3 0.5
84.1 1 0.1
66.9 0.2
45.8 + 1.8
67.8
SHOT-IM
61.5 1.7
84.6 1 0.3
68.0 1 0.0
33.8 - 0.3
62.0
SHOT
61.6 1.8
84.8
68.0 0.0
34.6 0.3
62.3
PL
6
50.1
1.5
36.8 - 4.4
60.4
PL-C
73.3 0.8
84.7 0.3
66.4
47.0 1 1.7
67.9
Tent-Full
75.4 0.6
$7.0 0.2
6..9
4..6
68.0
BN-Norm
71.3
85,8 0.1
6.14
42.3
66.5
Tent-C
72.4
84.4
6.22
42.4
66.4
Tent-BN
65.6 1.4
84.9 + 0.0
6.27
42.7 0.5
65.2
T3A (Ours)
76.0 + 0.3
85.1 1 0.2
44.6 14.60.99 0.9
68.5*
algorithms for training-phase (the third block of Table 1), and different backbone networks (Table 2). Note that this property is important in practice because a better backbone network usually give significant performance gains. For example, Table suggest practitioner should try HVIT, which outperforms ResNet50 by a large margin (7.8 points in average) if computational resources allow. T3A can boost HVIT's performance by 1.0 points. Unlike existing studies that update the model with SGD during testing, the proposed method is optimization-fres Therefore, the computational overhead negligible, and the behavior is unlikely to become unstable, making T3A especially suitable for online settings where the model needs to (adapt and) predict online with typical DG.
One the limitations T3A is how to extend it beyond the classification problem. Since the proposed method creates template online for each prediction class, it is not trivial to adapt it for continuous prediction. Note that Tent have the same problem, as prediction entropy is hard to compute in the regression
It is future task apply this idea broader range of problem settings.
Another potential drawback of the proposed method is that the model can change any time, making it difficult to thoroughly test its behavior in advance. This may raise ethical concerns in some sensitive applications, making it more difficult to sanitize the model and ensure does not make unfair decisions. In such situation, fully online adaptation might be difficult, and one may want to update the model offline. We encourage examination of each of these works on the frontier of test-time adaptation.
Although accuracy was greatly improved, there is massive room for improvement the performance in the unknown domain is still significantly worse than performance in the known domain. From probabilistic viewpoint, the templates of each class can be regarded the statisties of the P(Z
P(2) adjusting it can correlates the prediction. From this perspective, using the average templates might be too restrictive, and one can use higher-order statistics to improve performance. Alternatively, one can retain all reliable samples, approximating P(Z as empirical distribution. We hope that the findings of this paper will lead to better test-time adaptation method and lead to the development of machine learning systems that work well in unknown environments.
and our method adjusts it. As is connected to the prediction P(Y P(ZYY))())
10

---

Acknowledgements
This work has been supported by JSPS Grant-in-Aic for Early-Caree Scientists Number JPISK18 101 and the Mohammed bin Salman Center for Future Science and Technology for Saudi-Japan Vision 2030 at The University of Tokyo (MbSC2030). Computational resouree of Bridging Cloud Infrastructure (ABCT) provided by National Institute of Advanced Industrial Science and Technology (AIST) was used.
References
[1] K. Akuzawa, Yusuke Iwasawa, and Y. Matsuo. Adversarial invariant feature learning with accuracy constraint for domain generalization. In ECML/PKDD, 2019.
[2] Michael A. Alcorn, Qi Li, Zhitao Gong, Chengfei Wang, Long Mai, Wei-Shinn Ku, and Anh M Nguyen. Strike (with) pose: Neural networks are casily fooled by strange poses of familiar objects. CVPR, pages 4840 4849, 2019.
[3] Martin Arjovsky, Léon Bottou, Ishaan Guirajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907.02893, 2019.
[4] Y. Balaji, S. Sankaranarayanan, and R. Chellappa. Metareg: Towards domain generalization using regulaeguuaiizztion. In NewrIPS, 2018.
[5] Sara Beery, Grant Van Hor,, and P. Perona. Recognition in terra incognita. In ECCV, 2018.
[6] Gilles Blanchard, Gyemin Lee, and Clayton Scott. Generalizing from several related classification tasks to new unlabeled sample. In NIPS, pages 2178-2186, 2011.
[7] M. J. Choi, Joseph J. Lim, A. Torrala,, and A. Willsky. Exploiting hicrarchical context on a large database of object categories. CVPR, pages 129- 239-136,111 2010.
[8] Dengxin Dai and L. Gool. Dark model adaptation: Semantic image segmentation from daytime to nighttime. ITSC, pages 3819-3824, 2018.
[9] A. Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, M. Dehghani, Matthias Minderer, G. Heigold, S. Gelly, Jakob Uszkoreit, and N. Houlsby. An image worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021.
[10] Kurt Driessens, Peter Reutemann, Bernhard Pfahringer, and Claire Leschi. Using weighted nearest neighbor to benefit from unlabeled data. PAKDD, 2006.
[11] M. Everingham, L. Van Gool, C. K. 1. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results. http://www.paeal network.org/challn
[12] Chen Fang, Ye Xu, and Rockmore. Unbiased metric learning: On the utilization multiple datasets and web images for softening bias. ICCV, pages
2013.
[13] Li Fei-Fei, R. Fergus, and P. Perona. One-shot learning of object categories. IEEE Transactions on Puttern Analysis and Machine Intelligence, 28:594-611, 2006.
[14] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In JCML. 2015.
[15] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Lavio- lette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. JMRL, 171))2099-20300, 2016.
[16] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, M. Bethge, Felix Wichmann, and W. Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. ArXiv, 1..2/8811.12221.... 2019.
[17] Ishaan Guirajani and David Lopez-Paz. In search of lost domain generalization. In ICLR, 2021.
[18] Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. CVPR, pages 770-778, 2016.
[19] Timothy M. Hospedales, Antreas Antoniou, P. Micaelli, and A. Storkey. Meta leaanie in neural networks: A survey, IEEE transactions pattern analysis and machine intelligence, PP, 2021.
11

---

[20] Zeyi Huang, Haohan Wang, E. Xing, and Dong Huang. Self-challenging improves cross-domain general- ization. In ECCV, 2020.
[21] Yusuke Iwasawa, K. Akuzawa, and Y. Matsuo. Stabilizing adversarial invariance induction from divergence minimization perspective. In IJCAI, 2020.
[22] T. Joachims. Transductive inference for text classification using support vector machines. In ICML, 1999.
[23] Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICIRR, 2015.
[24] Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, J. Puigcerver, Jessica Yung, S. Gelly, and N. Houlsby, Big transfer (bit): General visual representation learning. In ECCV, 2020.
[25] Jogendra Nath Kundu, Naveen Venkat, V. RahulM., and R. Venkatesh Babu. Universal source-free domain adaptation. CVPR, pages 4543 4552, 2020.
[26] Dong-Hyun Lee. Pseudo-label: : The simple and efficient semi-supervised learning method for deep neural networks. 2013.
[27] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generalization. In ICCV, pages 5543-5551. IEEE, 2017.
[28] Da Li. Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Learning to generalize: Meta-leariing for domain generalization. In AAAI, 2018.
[29] Haoliang Li, Sinno Jialin Pan, S. Wang. and A. Kot. Domain generalization with adversarial feature learning. CVPR, pages 5400-5409, 2018
[30] Rui Li, Qianfen Jiao, Wenming Cao, H. Wong, and Si Wu. Model adaptation: Unsupervised domain adaptation without source data. CVPR, pages 9638-9647, 2020.
[31] Y. Li, Yongxin Yang, W. Zhou, and Timothy M. Hospedales. Feature-critic networks for heterogeneous domain generalization. In ICML, 2019.
[32] Li, Mingming Gong, Xinmei Tian, Tongliang Liu, and Dacheng Tao. Domain generalization via conditional invariant representations. In AAAI, 2018.
[33] Li, Xinmei Tian, Mingming Gong, Yajing Liu, Tongliang Liu, Kun Zhang, and Dacheng Tao. Deep domain generalization via conditional invariant adversarial networks. In ECCV, pages 624-639, 2018.
[34] Jian Liang, Hu, and Jiashi Feng. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In ICML, 2020.
[35] Thomas Mensink, Jakob J. Verbeek, Florent Perronnin, and Gabriela Csurka. Distance- bistance-based image classification: Generalizing to new classes at near-zero cost. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35:2624 2637, 2013.
[36] H. Nam, Hyunjae Lee, Jongchan Park, W. Yoon, and Donggeun Yoo. Reducing domain gap via style- agnostic networks. ArXiv, abs/1910. 11645, 2019.
[37] Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22: 12:1595-135, 2010.
[38] V. Patel, Raghuraman Gopalan, R. Li, and R. Chellappa. Visual domain adaptation: A survey of recent advances. IEEE Signal Processing Magazine, 32:53-65 09,33-6,,2200 2015.
[39] Sylvestre-AIie Alvise Rebufli, Alexander Kolesnikov, G. Sperl, and Christoph H. Lampert. icarl: Incremental classifier and representation learning. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5533-5542, 2017.
[40] Bryan C. Russell, A. Torralba, K. Murphy, and W. Freeman. Labelme: A database and web-based tool for image annotation. International Journal of Computer Vision, 77:157-173, 2007.
[41] Shiori Sagawa, Pang Wei Koh, Hashimoto, and Perey Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. ArXiv, 0ss/1911.0873.1 2019.
[42] K. Saito, Y. Ushiku, and T. Harada, Asymmetric tri-training for unsupervised domain adaptation. In ICML, 2017.
12

---

[43] Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringrmann, Wieland Brendel, and Maithias Bethge. Improving robustness against common comuptions by covariate shift adaptation. In NeurlPS, 2020.
[44] J. Snell, Kevin Swersky, and R. Zemel. Prototypical networks for few-shot learning, In NIPS, 2017.
[45] Baochen Sun and Kate Sacnko. Deep coral: Correlation alignment for deep domain adaptation. In ECCV Workshops, 2016.
[46] Y. Sun, Xiaolong Wang, Zhuang Liu, Miller, Alexei A, Efros, and Moritz Hardt. Test-time training for out-of-distributioo generalization. ArXiv, abs/1909. 13231, 2019.
[47] Tolstikhin, Houlshy, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung. Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and A. Dosovitskiy. Mlp-mixer: An all-mip architecture for vision. ArXiv, abs/2105.01601, 2021.
[48] Hugo Touyron, M. Cord, M. Douze, Francisco Massa, Alexandre Sablayrolles, and Hery's e J'egou. Training data-efficient image transformers & distillation through attention. ArXiv, abs/2012. 2020.
[49] V. Vapuik. Statistical learning theory. 1998.
[50] Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and S. Panchanathan. Deep hashing network for unsupervised domain adaptation. CVPR, pages 5385 5354-5394,25 2017.
[51] G. Volk, Mueller, Alexander von Beruuth, Dennis Hospach, and O. Bringmann. Towards robust enn-based object detection through augmentation with synthetic rain variations. /TSC, pages 285-292, 2019.
[52] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Olshausen, and Trevor Darrell, Tent: Fully test-time adaptation by entropy minimization. In ICLR, 2021.
[53] Yufei Wang. Haoliang Li, and Kot. Heterogeneous domain generalization via domain mixup. ICASSP, pages 3622-3626, 2020.
[54] Ross Wightman. Pytorch image models. https://githhb.. https://gilubb.oo//vvignan con//vigtmmm/ppttoccimnee models, 2019.
[55] Garrett Wilson and D. Cook. A survey of unsupervised deep domain adaptation. TIST, 11:1 46, 2020.
[56] Minghao Xu, Jia yu Zhang, Bingbing Ni, Teng Li, Chengjie Wang, Qi Tian, and W. Zhang. Adversarial domain adaptation with domain mixup. In AAAI, 2020.
[57] X. Xu. Timothy M. Hospedales, and S. Gong. Transductive zero-shot action recognition by word-vector embedding. International Journal of Computer Vision, 123:309-333, 2016.
[58] Shen Yan. Huan Song, Nanxiang Li. L. Zou, and Liu Ren. Improve unsupervised domain adaptation with mixup training. ArXiv, abs/2001 00072000000077, 2020.
[59] Hongyi Zhang, Moustapha Cissé, Yann Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. ArXiv, abs/1770.09412, 2018
[60] Marvin Zhang, H. Marklund, Abhishek Gupta, Sergey Levine, and Chelsea Finn. Adaptive risk minimiza- tion: A meta-learning approach for tackling group shift. ArXiv, abs/2007.0290 2020.
[61] K. Zhou, Z. Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. ArXiv, abs/2103.02503, 2021. 2021.
Checklist
1. For all authors...
(a) Do the main claims made the abstract and introduction accurately reflect the paper's 's contribu- tions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 5.1.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See Section 5.1.
(d) Have you read the ethies review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
13

---

3. If you experiments...
(a) Did you include code, data, and instructions needed to reproduce main experimental results (either in the supplemental material or as URL)? [Yes] See supplemental materials.
(b) Did you specify all the training details (c.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section and Appendix A.
(c) Did you report error bars (e.g.. with respect to the random seed after running experiments multiple times)? (Yes) We repeated entire experiments three times as recommended by the prior works.
(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, cloud provider)? [Yes] See Section 4 and Appendix A.
4. If you are using existing assets (e.g.. code, data, models) or curating/relessing new assets...
(a) your work existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes] See Appendix A.
(c) Did you include any new assets either in the supplemental material or as URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're 'r us-
ing/curating? [Yes]
(c) Did you discuss whether the data you are using/curating contains personally identifiable informa- tion offensive content? [Yes]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
14


Reviews:
[{"review_id": "_DI1o13yGcO", "reviewer": "Reviewer_rrrY", "summary": "This paper proposed a test-time adaptation (TTA) solution for domain generalization (DG) problems. It exploits prototypical representations, which are used to classify test samples and are refined over time. The method performs well, when compared with standard DG methods and the recently introduced TTA method \"TENT\".", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n\n- The method proposed method is reasonable, and scores favorably with competing algorithms in standard DG benchmarks.\n\n- The paper is overall well written, with Figures well designed and properly placed. \n\n- The hyper-parameter selection is transparent, also for what concerns the competing algorithm. The plots in Figure 2 are very helpful in assessing the robustness of the proposed method with respect to the baseline across different runs.\n\nWeaknesses\n\n- According to Lines 167/168, the prototypes are refined every time a new test sample is processed: this is consistent with Wang et al. ICLR 2021 (TENT), but it brings an advantage over standard DG methods that the method is compared against - which make their decision by evaluating single samples independently. It would be good to include also some results where the prototypes are not updated, to have a fairer comparison with DG methods. For what concerns the protocol followed in this work and by Wang et al. ICLR 2021, it would be good to also include a comparison against Schneider et al. ICLR 2021 (\"Improving robustness against common corruptions by covariate shift adaptation\"): it is essentially a simpler version of TENT, where only the BN statistics is updated.\n\n- I believe the comparison with TENT (and with the suggested Schneider et al., ICLR 2021) should be carried out by using a standard ResNet with BN: I am not fully convinced that the precautions taken (lines 246/247) put the competing algorithms in a favorable starting point. It should be fairly simple to test the proposed method with a ResNet-18/50 with BN, am I correct? In this way, the comparison with such related work (the most related to the proposed method) would utmost fair.\n\nAdditional comments\n\n- Line 162: why is the support set initialized with the weights of the network, and not with the average between the source features? In Eq.3 new features are included in the support set, and in Eq.5 the average between the elements of the support set is carried out: from the latter Eq., it seems that they are all feature vectors, without weight vectors. Is it a typo or am I getting this wrong?\n\n- The motivation in lines 126-129 are not very strong: it is true that test-time adaptation is yet to be used in DG, but it was tested in very related problems - for instance, under corrupted test samples or natural domain shifts.\n\n- In the related work, it should be included [Chidlovskii et al., \"Adaptation in the Absence of Source Domain Data\", SIGKDD 2016] for what concerns source-free adaptation, which I believe it is the first work tackling this problem formulation.\n\n- Researchers started exploiting prototypical representations for classification before they were used in the cited few-shot learning works. See, e.g., Mensink et al., \"Distance-based image classifi- cation: Generalizing to new classes at near-zero cost\" TPAMI 2013.\n\n- Lines 253-258, I think [47] should be [16]\n\n- I do not believe that Remark 1  in Sec.3.2 is relevant: it is widely accepted that finetuning the final layer carries good performance in general.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "/"}, "extra_text": {}}, {"review_id": "JuXdOoHLQG", "reviewer": "Reviewer_3ToC", "summary": "This paper introduces one algorithm for domain generalization and test-time template adjuster (T3A) for developing a model that performs well under conditions that are different from training conditions. The designed T3A is composed with two procedures: (1) compute a pseudo prototype representation for each class based on unlabeled online dataset, then (2) classify the data based on its distance to the pseudo prototypes. The proposed T3A can improve the performance on the unseen domains, and also outperform the state-of-the-arts.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n- Currently, domain generalization is a hot and important topic. Many works only focus on the training phase. However, T3A focuses on altering the testing phase based on the existing model, which is practical in the realistic world. \n- The proposed method in algorithm 1 is simple and easy to implement. \n- There is no training on feature extractor parameters, so the training is so fast.\n\nCons:\n- In the proposed method, the feature extractors trained in each domain are fixed, and the goal is to solve a semi-supervised problem on a sample/prototype basis. Although not in the context of domain generalization, the reviewer thinks that this kind of problem setting was discussed before deep learning became popular. See, for example, reference [a].\n-- [a] Driessens K., Reutemann P., Pfahringer B., Leschi C. Using Weighted Nearest Neighbor to Benefit from Unlabeled Data. PAKDD, 2006.\n- A survey of sample/prototype-based semi-supervised learning methods prior to deep learning and a discussion of the superiority over these methods is needed.\n- Except for the context of domain generalization, the novelty of the proposed method seems to be unclear.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "- The overall algorithm utilizes meta learning for boosting the transferability between the source and target domain. But based on the current experiment, the proposed method can perform well on the close-set dataset. I’m wondering if this method can perform well on the partial domain adaptation.\n- The paper also mentions that the template trained by source data can’t guarantee that it is a good template for the target dataset. I’m also wondering to what extent that the template on the source data can reflect the feature of the transferred information?\n- The format of section 5 is wired. If there is only one subsection, it doesn’t have to be one subsection.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "0Fog_pGr1Hm", "reviewer": "Reviewer_mpea", "summary": "This paper proposes a new online method for test-time adaptation to domain shift. Specifically, the final layer of a network (i.e. the classifier) is adapted to test data using pseudo-prototypes, which combine the ideas of pseudo-labelling [1] and prototypical representations [2]. That is, pseudo-labels define the support set for each class, and the centroids of these support sets are the prototypes. Experiments demonstrate incremental but reliable performance improvements in the test domain using this optimization-free method.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Summary**: Online test-time domain *adaptation* is an interesting and useful research direction, overcoming many of the impracticalities of domain *generalization* (no updating on the test data). The authors’ combination of two existing ideas -- pseudo-labelling [1] and prototypical representations [2] -- seem effective for this setting. The improvements are incremental but consistent. I believe the paper is just above the acceptance threshold, and would be improved with the following: (i) stronger baselines (see below); and (ii) a comparison to “offline” optimization-based methods [3,4] when using two different model selection strategies---training-domain validation-set and test-domain validation-set (oracle) validation sets (see below).\n\n**Originality**: A new combination of well-known techniques for the purpose of *online* test-time adaptation. Related work is adequately cited, making clear how this work differs from source-free domain adaptation (offline vs. online) and prototypical networks (assume access to a few labels in the test domain vs. pseudo-labels).\n\n\n**Quality**: Overall, the submission is technically sound, the claims well supported, and the methods appropriate. However, the empirical evaluation could be improved with:\n- *Additional baselines*\n     - *Full Tent method*: Tent-C and Tent-BN are very weak baselines -- they are quite far from the proposed Tent method [4]. In addition, many networks *do* still use batch normalization, including some of the backbones used in this paper (e.g. ResNet50). Thus I think a comparison to the actual Tent method (i.e. updating all BN parameters) is warranted, e.g. for the ResNet models. I think the difficulty of model selection will ensure the proposed method still “wins”, but the comparison is important.\n     - *SFDA methods in an online setting*: Do they do poorly? They seem like necessary baselines to show the advantages of the proposed method (e.g. better sample-efficiency due to less updates, or better performance due to easier model selection). E.g. use SHOT[3] with 200 examples per test batch, rather than the current 32. This is the online setting of Tent [4].\n     - *Pseudo-labelling*: could the final layer be updated with pseudo-labelling to isolate the benefits of: (i) prototypes vs. pseudo-labelling; and (ii) being optimization-free (easier model selection)?\n- *Model selection*: This is a huge advantage of the proposed method over optimization-based methods (e.g. Tent or SFDA methods). Thus, I think it warrants a little more emphasis / analysis:\n     - *Lines 261--262*: \\emph{These parameters must be selected before the deployment}\n     - Compare the performance of your method to these optimization-based methods when using: (i) training-domain validation-set; and (ii) test-domain validation set (oracle). This would very clearly illustrate the limitations of these methods, and thus the advantage of the proposed one. E.g. a figure similar to Figure 1a for different model selection methods.\n     - *Figure 1c*: I think this figure is a bit unfair/misleading as: (i) Tent-C is used, rather than the actual Tent method[4], despite ResNet50 having BN layers; and (ii) the poor entropy-reduction of Tent-C is likely explained by poor model selection (training-domain validation set), and this point is only very briefly noted in lines 208--209---not in the caption.\n\n**Clarity**: The submission is clearly written and well organized.\n\n**Significance**: The importance and usefulness is relatively clear. The proposed method improves the state-of-the-art for *online* test-time adaptation on networks with *no batch-normalization*. The significance and usefulness would be much improved by: (i) comparing to the actual Tent method [4] on the resnet models which use BN; and (ii) illustrating the poor suitability of existing *offline* SFDA methods to this “online” problem, and as a result, the need for the proposed method.\n\n**Additional feedback**: \n- *A model can be confident but wrong.* The authors seem to assume that this never happens. E.g. on line 128, “more confidence predictions are more correct” -- this needs to be weakened, e.g. “more confident predictions are often more correct”... but not always. Adversarial examples are one… example, but I’m sure it can happen even when not hand-crafted. I think this is a minor limitation worth acknowledging (also applies to other entropy-reduction methods too).\n- *Domain generalization vs. domain adaptation.* I am not sure I agree with calling this a domain *generalization* method, rather than an online method for source-free domain *adaptation*. DG usually implies *no updating* on the test data.\n\n\n\n[1] Lee, D. H. (2013). Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In *Workshop on Challenges in Representation Learning, ICML*, 3, pp. 896.\n\n[2] Snell, J., Swersky, K., & Zemel, R. (2017). Prototypical Networks for Few-shot Learning. In *Advances in Neural Information Processing Systems*, 30, pp. 4077-4087.\n\n[3] Liang, J., Hu, D., & Feng, J. (2020). Do we really need to access the source data? Source hypothesis transfer for unsupervised domain adaptation. In *International Conference on Machine Learning*, pp. 6028-6039.\n\n[4] Wang, D., Shelhamer, E., Liu, S., Olshausen, B., & Darrell, T. (2020). Tent: Fully Test-Time Adaptation by Entropy Minimization. In *International Conference on Learning Representations*.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes, the authors have addressed the limitations and potential ethical concerns.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper is proposing a test-time adaptation method for domain generalization. The proposed idea is quite simple and effective. It uses the predictions as pseudo labels and only updates the final classification layer without any gradient-based optimization. Hence, it is not only simple and effective but also lightweight. All reviewers agreed that the proposed method has merits; however, there were some concerns with the empirical study. The authors provided significant additional empirical support during rebuttal and reviewers appreciated it. I believe it is a solid paper that will likely have a significant impact on the community.
Decision: Accept (Spotlight)