Test-Time Classifier Adjustment Module for Model-Agnostic Domain Generalization
Year: 2022

Abstract:
This paper presents a new algorithm for domain generalization (DG), test-time template adjuster (T3A), aiming to robustify a model to unknown distribution shift. Unlike existing methods that focus on training phase, our method focuses test phase, i.e., correcting its prediction by itself during test time. Specifically, T3A adjusts a trained linear classifier (the last layer of deep neural networks) with the following procedure: (1) compute a pseudo-prototype representation for each class using online unlabeled data augmented by the base classifier trained in the source domains, (2) and then classify each sample based on its distance to the pseudoprototypes. T3A is back-propagation-free and modifies only the linear layer; therefore, the increase in computational cost during inference is negligible and avoids the catastrophic failure might caused by stochastic optimization. Despite its simplicity, T3A can leverage knowledge about the target domain by using off-the-shelf test-time data and improve performance. We tested our method on four domain generalization benchmarks, namely PACS, VLCS, OfficeHome, and TerraIncognita, along with various backbone networks including ResNet18, ResNet50, Big Transfer (BiT), Vision Transformers (ViT), and MLP-Mixer. The results show T3A stably improves performance on unseen domains across choices of backbone networks, and outperforms existing domain generalization methods.

heading: 1 Introduction
text: Deep neural networks often fail to generalize to out-of-distribution samples. Accuracy suffers when the model performs under conditions different to those of training, such as variations in light [8], weather [51], object poses [2], textures [16], or object backgrounds [5]. Nevertheless, the model may be deployed to different conditions in practical situations; thus, some countermeasures are needed.
Over the past decade, various studies have focused on training a generalizable model to unseen domains given a dataset consisting of several source domains. This setting is usually denoted as domain generalization (DG) [6, 61]. Domain generalization operates under the assumption that one can improve robustness to domain shift by incorporating the structure common to multiple domains. For example, domain-invariant feature learning constrains the representation to be invariant to domain shifts [14, 45, 29]. Other methods use meta-learning [19] to learn how to regularize the model to improve the robustness [28, 4, 31]. However, despite significant work on this front, machine learning systems are still vulnerable to domain shifts even after using the above methods during training. Notably, recent large-scale benchmarks [17] show that many approaches do not provide significant improvement compared to simple supervised learning, i.e., empirical risk minimization (ERM), with a proper and practical experimental setup. It suggests that the setup in its current state may be too difficult, and a different approach might be needed.
This paper proposes a method of using additional off-the-shelf data in the DG setup, the unsupervised data available at the test-time. Since no data about the target domain is available during training in
35th Conference on Neural Information Processing Systems (NeurIPS 2021), virtual.
a DG setup, the existing domain generalization algorithms focus on how to use labeled data from multiple-source domains. However, at test-time the model always has access to test data from the target domain. Although the available data is constrained to be (1) unlabeled and (2) only available online (models can not know all test cases in advance), this data provides clue about the target distribution that is not available during training. It is natural to ask the question: How can we use the off-the-shelf unlabeled data available at test-time to increase performance on the target domain?
It is worth emphasizing that our setting is different from the transductive setting [49, 22, 57] where all test cases are known in advance, even though we use test data for adjustment. When testing, the model is usually deployed in some environment, and must work well on various samples that will appear continuously. Similarly, the deployed model usually needs to make correct predictions at that moment; there is no point in going back in time and correcting the predictions. Therefore, it is desirable that adjustment and inference be performed at the same time, not offline after a large amount of data has been accumulated. Looking beyond domain generalization, some recent studies suggest optimizing the model during test time using objective function defined only by unsupervised data (e.g, prediction entropy) [34, 52]. However, updating parameters using stochastic gradient descent (SGD) increases computational costs and harm inference throughput. In addition, data available at test time is limited, and stochastic optimization can lead to catastrophic failure.
To this end, we present test-time templates adjuster (T3A), which adjusts the linear classifier (the last layer of deep neural networks) at test-time. T3A adjusts the weights of the linear classifier as the following optimization-free procedure: (1) create a pseudo-prototype for each class using online unlabeled data and the classifier trained in the source domains, (2) and then classify each sample based on its distance to the pseudo-prototype. This procedure makes the adjusted decision boundary avoid the high-data density region on the target domain and reduce the ambiguity (entropy) of predictions, which is known to be connected to classification error [52]. Since T3A does not alter the training phase, it can be used together with existing DG algorithms. Moreover, it can be used together with any classification model since it only adjusts the linear classifier on top of the representations. Some readers may wonder how effective it is to modify only the linear classifier while freezing the representation itself. Later in this paper (Section 3.2), we empirically demonstrate that this modification is indeed beneficial.
We evaluate our method on multiple standard domain generalization benchmarks, namely VLCS [12], PACS [27], OfficeHome [50], and TerraIncognita [5]. We compare our method with (1) various DG algorithms reported in [17] and (2) Tent [52] that minimizes the prediction entropy at test time using SGD. With the standard ResNet50 backbone [18], T3A improves ERM by 1.5 points on average accuracy over four dataset, and outperforms most existing DG algorithms. Furthermore, we evaluated our method with 10 different backbone networks, including residual networks (resnet18 and resnet50), big transfer (BiT-M-R50x3, BiT-M-R101x3, and BiT-M-R152x2 [24]), vision transformers (ViT-B16, ViT-L16, Hybrid ViT [9], DeiT [48]), and MLP-Mixer (Mixer-L16) [47]. The results show that T3A gives a statistically significant performance gain against ERM on all backbone networks.

heading: 2 Preliminary and Related Work

heading: 2.1 Domain Generalization
text: Problem setup Following [6], we assume multiple datasets Dd = {(xdi , ydi )} nd i=1 collected from several different domains d ∈ {1, · · · , dtr}. The dataset Dd from domain d contains identically and independently distributed samples characterized by some probability distribution P d(X,Y ), where X and Y are random variables of input and target, respectively. Then, our goal is to develop a predictor f(X) that performs well on some unseen test domain, which is characterized by a different probability distribution P (X,Y ) 6= P d(X,Y ) for all d ∈ {1, · · · , dtr}. Note that one can not assume the target distribution during training, e.g., no data about the target distribution is available at the time. Therefore, the predictor is usually trained on datasets from several source domains. For example, the predictor can be trained by minimizing the empirical risk:
argmin φ
1
d dtr∑ d=1 1 nd nd∑ i=1 `(f(xdi ), y d i ), (1)
where φ is the set of the parameters of the function, and ` is a loss function measuring prediction error. In the rest of this paper, optimizing the predictor with eq. 1 is called ERM. In a real application,
the model will be deployed after training and expected to classify the data in an online manner. For benchmarking the algorithm, given a dataset containing nd domains, we usually use the leaveone-domain-out procedure, which uses a single domain as a test domain and the others as training domains. The procedure is repeated nd times, changing the test domain every time.
Algorithms A central branch of DG algorithm is domain-invariant feature learning, which explicitly reduces the domain gaps on a space of latent representations. For example, [14] proposed domainadversarial networks (DANN) which measure the domain gaps via an external domain classifier. CORAL [45] align the second-order statistics of representations among different domains. [29] uses maximum mean discrepancy (MMD) to measure the domain gap. Many extension have been proposed [33, 1, 21], but they are all the same in that they enhance domain invariance. Another branch are meta-learning-based methods [28, 4, 31], which divide the available domains into meta-train-domains and meta-test-domain and regulate the model trained in meta-train-domains to be useful for the meta-test-domain. Invariant risks minimization [3] regularizes ERM with a gradient normalization penalty over a dummy classifier. Several studies propose to augment the data using mixup [59] between two source domains, which implicitly enhances invariance to domain shifts [56, 58, 53].
Key differences As briefly mentioned above, existing domain generalization algorithms focus on the training phase; how to regularize the predictor using the knowledge from multiple source domains. Our work focuses on the test phase; how to adjust the model using online and unlabeled data, which can characterize the target distribution. Note that proposed method works fully online; It does not require access to offline unlabeled data, and therefore can be compared fairly with existing DG methods

heading: 2.2 Other Related Work
text: Unsupervised domain adaptation. Our work is related to unsupervised domain adaptation (UDA) [37, 38, 55] as both methods aim to adapt a model given unsupervised data. However, our work primarily differs from UDA in that UDA focuses on adapting during training, while we focus on adapting during testing. In other words, UDA assumes we can access labeled data from the source domain and (unlabeled) data from the target domain at the same time, which is not always possible.
Source-free domain adaptation. Among them, recent “source-free” setups are particularly similar to our setting [30, 25, 34]. In these setups, source data is not needed during the adaptation phase, and the model is adapted using the unlabeled data solely from the target domain. However, this adaptation is usually made in an offline manner, i.e., these source-free methods optimize offline with multiple losses for multiple epochs. Our method adjusts the classifier in an online-manner, and therefore is suitable for a domain generalization setup where the trained model is assumed to be deployed.
Test-time adaptation. Regarding the problem setup, our work is most closely related to testtime adaptation [52] or test-time training [46]. Notably, [52] proposes fully test-time adaptation, which modulates the BN parameter by minimizing the prediction entropy using stochastic gradient descent. The concept of test-time adaptation is very similar to our work and can be used in domain generalization, however, it has not been fully investigated under the domain generalization setup. Moreover, recent architectures do not employ batch normalization either on pre-training (mainly to avoid the large memory usage required by BN) or fine-tuning phase (for improving performance). Besides, minimizing the prediction entropy using SGD could lead to trivial solutions, such as being biased to predict only a particular class. In comparison, (1) our method can be used together with any classification models since it only adjusts the linear classifier on the top of the representations, (2) our method alleviates catastrophic forgetting due to not using SGD during test-time.
Prototypical networks. Before deep learning become popular, the prototype-based classifier is well-investigated in the context of semi-supervised learning [10], continual learning [35, 39], and few-shot learning [44]. Our work is most similar to prototypical networks [44], which combine prototypical classifier and deep neural networks as with our method. However, the use-cases mentioned above of prototypical networks assume access to a few labeled data from the same domain, which differs in our case. To handle the difference, we combine prototypical networks with pseudolabeling techniques, which are often used in domain transfer literature [42]. Besides, connection to entropy minimization is a new perspective introduced in this paper.
Algorithm 1 Algorithm of T3A for prediction.
Input: Feature extractor fθ, the batch of input B, and support sets Sk available at this point. Output: Prediction for all x ∈ B, where x ∼ P (X).
# Step1. Adjust the template for each class using the B. for x ∈ B do ŷ = argmax qω(Y = yk|fθ(x)) (eq. 2) Sk = Sk ∪ { fθ(x)‖fθ(x)‖} for y
k = ŷ (eq. 3) end for Filter support sets with eq. 6 # Step2. Predict based on the distance between the adjusted template. return argmax γ(Y = yk|fθ(x)) for all x ∈ B (eq. 4)

heading: 3 Proposal: Optimization-Free Test-Time Classifier Adjustment Module
text: We propose to replace the output layer of the predictor trained on source-domains (i.e., linear classifier) to a pseudo-prototypical classifier, whose prototype features are adjusted during test time while fixing the features already trained on the source domains. We call our method T3A, for Test-Time Templates Adjuster. We first explain the detailed algorithm (Section 3.1), and explain how and why it works (Section 3.2). Algorithm 1 outlines the procedure.

heading: 3.1 Algorithm
text: We assume the predictor is deep neural networks (DNN) obtained by some learning algorithm (e.g., ERM, DANN, CORAL, etc.) using data from source domains. For convenience, the entire DNN is divided into a linear classifier qω for the last layer and a feature extractor fθ for the rest, where ω and θ are the parameters of neural networks. In usual domain generalization setup, fθ and qω are used to predict data from the test domain. For new data x, the prediction is given by taking the argmax over the following approximated probability distribution:
argmax yk qω(Y = yk|fθ(x)) = exp(z · ωk)∑ j exp(z · ωj) , (2)
where z = fθ(x), ωk ∈ Rzdim is the k-th element of the weights matrix in ω, and zdim is the dimension of z, which depends on the feature extractor fθ. In this prediction, ωk works as the template of representation for the class k, and prediction is done by measuring the distance (dot product) between the template and the representation of the input data. Since this template was trained in the source domain, there is no guarantee that it will be a good template in the target domain.
T3A adjusts the templates during test-time. Assume we have (batch of) test-data x at time t drawn from target distribution P (X) 6= P d(X) for all d ∈ {1, · · · , dtr}. As each prototype should be related to some class, we first augment the input data x via pseudo label ŷ, which is obtained via eq. 2. Then, we update a support set Skt as follows:
Skt =
{ Skt−1 ∪ { fθ(x) ‖fθ(x)‖} if ŷ = y k
Skt−1 else, (3)
where ‖a‖ represents the L2 norm of the vector a and Sk0 = { ω k
‖ωk‖}. If the input data contains multiple samples at the same time (e.g., a batch of data), the above procedure is repeated for each sample in the batch. Then, prediction is done by taking the argmax over the following adjusted probability distribution:
argmax yk γc(Y = yk|fθ(x)) = exp(z · ck)∑ j exp(z · cj) , (4)
where ck are the centroids of Sk: ck = 1 |Sk| ∑ z∈Skt z. (5)
VLCS PACS OfficeHome Terra Dataset
0.0
0.1
0.2
0.3
0.4
En tro
py H
q( y)
Seen Unseen
(a) Prediction entropy
VLCS PACS OfficeHome Terra Dataset
0.0
0.1
0.2
0.3
0.4
En tro
py H
q( y)
ERM +T3A +Tent-C
(b) ERM vs. T3A vs. Tent
VLCS PACS OfficeHome Terra Dataset
0.0
0.2
0.4
0.6
0.8
1.0
Ac cu
ra cy None
+head +body +all
(c) Transferability
Figure 1: Pre-experimental results. (a) Comparing the entropy of predictions (with ResNet50) on the source domains and the target domain. The domain shift increases entropy. (b) T3A effectively reduces entropy. (c) Transferability of each components of the model trained by ERM in each dataset.
Note that, this procedure will be repeated each time new data arrives, without discarding the support sets, which may make infeasible to retain all past data as in eq. 3. Also, some pseudo-labels are assigned to the wrong class, and using this data is not desirable as it adds noise to the templates and may deteriorate performance. To avoid this issue, we use the prediction entropy Hω(Ŷ |z) = − ∑
k qω(Ŷ = y k|z) log qω(Ŷ = yk|z) to filter unreliable pseudo-labeled data. Specifically, before
making a prediction using eq. 4, only a part of the support set is restored as follows:
Skt = {z | z ∈ Skt , Hω(Ŷ |z) ≤ αk}, (6)
where αk is the M -th largest entropy of the support set Skt (M is a hyperparameter).

heading: 3.2 Remarks
text: Remark 1: T3A implicitly reduces prediction entropy. As prior works suggest [52], the prediction entropy is often related to an error, as more confident predictions tend to be more correct. Figure 1-a shows that the prediction entropy also characterizes the difficulty in DG setup; entropy in the unseen domain tends to be greater than entropy in the seen domains. To be more specific, we first trained ResNet50 on the source domain by ERM. The training was done in leave-one-domain-out manner, and we conducted three experiments with a different seed each time. We used four standard datasets in domain generalization (VLCS [12], PACS [27], OfficeHome [50], and TerraIncognita [5]). We used the implementation of DomainBed [17], and used the default hyper-parameters for pre-training on source-domains and fine-tuning on a target domain: namely, we use Adam [23] with a learning rate of 5e-5 for optimization and use a batch size of 32 with no dropout or weight decay.
With this in mind, existing studies have modified the model parameters to explicitly reduce entropy. Although the proposed method does not explicitly reduce entropy, it has the effect of implicitly reducing it. This is because the proposed method uses a template updated with samples from the target distribution, which provides a decision boundary that avoids the dense parts of the target distribution. Figure 1-b compares the prediction entropy on the target domain among (a) ERM without test-time modulation, (b) T3A, (c) Tent-C, which updates the classifier to minimize entropy. The results show that T3A can effectively reduce entropy without using online optimization. Note that changing a hyper-parameter (such as learning rate) might change the results for Tent-C, but it may corrupt the entire classifier. See Section 4 for the hyperparameter selection.
Remark 2: T3A is computationally light. Unlike Tent, our method does not use SGD. Besides, the representations are fixed, and it is not necessary to repeat the forward propagation of the feature extractor. The only computational overhead is the cost of one forward propagation of the last linear layer, which is usually negligible compared to the forward and back propagation of feature extractors. Since it is not desirable to reduce throughput when considering online prediction, the proposed method is suitable in this respect as well.
Remark 3: Adjusting the linear classifier can significantly improve performance. Some readers may wonder how effective it is to modify only the linear classifier while freezing the representation. To answer this question, we compare the DG performance (None) before fine-tuning,
(head) after fine-tuning only qω , (body) after fine-tuning only fθ , and (all) after fine-tuning the entire network (Figure 1-c). Each block corresponds to a different dataset, and each color represents a different fine-tuning strategy. The results show that fine-tuning only the classifier often significantly improves performance. For example, in VLCS, the average performance score jumps from 72.5 to 82.9, which is close to the 84.7 obtained when the entire network is fine-tuned. The performance gain differs for each dataset, but the tendency is generally the same. In addition, this tendency was the same for other backbone networks including BiT, ViT, and Mixer (see Appendix B). These results indicate that adjusting only the linear classifier can significantly improve performance in various configurations. Note that the number of parameters of the linear layer is much smaller than those of the feature extractor in standard network architectures.

heading: 4 Experiment
text: We evaluate T3A on four standard domain generalization benchmarks, namely VLCS [12], PACS [27], OfficeHome [50], and TerraIncognita [5]. Our implementation uses the DomainBed library [17]1. We modify DomainBed (1) to use various backbone networks using the timm library [54]2, and (2) to implement test-time adaptation algorithms (ours and Tent). For Tent, we used the original implementation3. We run our experiments mainly on cloud V100x4 or A100x8 instances, depending on the memory usage of the backbone networks. See Appendix A for more information, including licensing information and total amount of compute.
Datasets. VLCS [12] comprises four photographic datasets d ∈ {Caltech101[13], LabelMe[40], SUN09[7], VOC2007[11]}, containing 10, 729 examples of 5 classes. PCAS [27] comprises four domains d ∈ {art, cartoons, photos, sketches}, containing 9, 991 examples of 7 classes. OfficeHome [50] includes domains d ∈ {art, clipart, product, real}, containing 15, 588 examples and 65 classes. TerraIncognita [5] includes photo of wild animals taken by camera at different locations. Following [17], we used datasets of d ∈ {L100,L38,L43,L46}, containing 24, 788 examples and 10 classes.
Backbone networks. For the main experiments, we use residual networks with 50 layers (ResNet50), which was the default setting of the prior studies. In addition, we tested our algorithms on 10 different pre-trained models: residual networks with different layers (ResNet18 and ResNet50), Big Transfer [24] with different layers (BiT-M-R50x3, BiT-M-R101x3, and BiT-MR152x2 ), Vision Transformers [9] with variations (ViT-B16, ViT-L16, HViT, which uses ResNet50 as patch embedding of ViT, DeiT [48]), and MLP-Mixer (Mixer-L16) [47].
Baselines. We compare our method to domain generalization algorithms and test-time adaptation algorithms. For domain generalization algorithms, we mainly compared with the results reported in [17]. These results include the following algorithms: Empirical Risk Minimization (ERM), Group Distributionally Robust Optimization (GroupDRO) [41], Inter-domain Mixup (Mixup) [56, 58, 53], Meta-Learning for Domain Generalization (MLDG) [28], DomainAdversarial Neural Networks (DANN) [15], Class-conditional DANN (C-DANN) [32], Deep CORrelation ALignment (CORAL) [45], Maximum Mean Discrepancy (MMD) [29], Invariant Risk Minimization (IRM) [3], Adaptive Risk Minimization (ARM) [60], Marginal Transfer Learning (MTL) [6] Style-Agnostic Networks (SagNet) [36], and Representation Self Challenging (RSC) [20].
We also compared our method with existing test-time adaptation methods. Note that we can not simply use BN-based methods (including Tent [52], which is the most up-to-date method) on the DG setup because [17] omit the BN layer from pre-trained ResNet when fine-tuning on source domains. Besides, several backbone networks evaluated in our paper do not contain BN from the beginning. Therefore, we first tested two slightly modified versions of Tent on standard DG setup (Table 1, Table 2, and Figure 2). Specifically, Tent-C modulates the entire classifier to reduce prediction entropy. Tent-BN adds one BN layer just before the linear classifier and then modulates BN’s normalization and transformation parameters. We then compare T3A with other test-time adaptation methods (including SHOT [34], pseudo labeling (PL) [26], Tent-Full [52], BN-Norm [43]) using ResNet18 and ResNet50 without removing batch normalization layer (Table 3).
1https://github.com/facebookresearch/DomainBed 2https://github.com/rwightman/pytorch-image-models 3https://github.com/DequanWang/tent
Hyperparameters and model selection. As [17] claimed, model selection is not trivial in DG and significantly affects performance. We used standard training-domain validation for selecting hyperparameters, which uses the subset of each training domain to choose a model. As reported in [17], we split the data from each domain into 80% and 20% splits and use larger splits for training and smaller splits to select hyperparameters. Following [17], we conduct a random search of 20 trials over a joint distribution of all hyperparameters to train the base model (see Appendix A.4).
In addition, T3A has one hyperparameter M for deciding the number of supports to restore, and Tent has two primary hyperparameters: β for multiplying the base learning rate (used for the base model) and γ for the number of iterations per adaptation. It is worth emphasizing that these parameters should be selected before the deployment, i.e., before accessing the test data. We simply selected these hyperparameters by the average accuracy in the training-domain validation data when using these adjustment modules. Specifically, we tested M ∈ {1, 5, 20, 50, 100,N/A} for T3A, where N/A means restoring all samples, and combination of β ∈ {0.1, 1.0, 10.0} and γ ∈ {1, 3} for Tent.

heading: 4.1 Results
text: Table 1 summarizes the results when ResNet50 is used as the backbone network. The first block (from ERM to RSC) is the value taken from [17]. The lines labeled ERM† and CORAL† are the scores reproduced in our environments. The proposed method and Tent are based on this reproduced model. Figure 2 shows the distribution of performance improvement by the proposed method for models trained with different hyperparameters (20× 3 for each test environment). In addition, Table 2 show the DG accuracy with 10 different backbones. Note that this experiment is conducted only on the default hyperparameter of ERM. Every number we report is a mean and standard error over three repetitions with different weight initialization, and dataset splits.
T3A stably improves the performance of the base model. The second block of Table 1 shows that T3A stably improves the performance of the ERM model. Specifically, the proposed method improves 2.3 points, 2.0 points, 1.9 points, and 0.5 points for each dataset respectively. The average
improvement from ERM is 1.5 points. For clarification, paired t-test was performed using 48 paired data (4 datasets, 4 test domains, 3 different seeds). As a results, the difference is statistically significant (p ≤ 0.01). Note that, Tent-BN improves the performance in PACS and OfficeHome, but it is not stable may be due to the failure of the optimization. Tent-C never improve the performance.
In addition, Fig. 2 show that the third-quarter quantile of the improvement range is often more than 0, which means the proposed method is also robust to the hyperparameters of the base model. Furthermore, Table 2 shows that the proposed method can also improve the performance of more sophisticated backbone networks. For example, the proposed method improves performance by an average of 1.0 points for HViT, which achieved the best performance of all backbones. The
improvement by the proposed method is statistically significant (p ≤ 0.05 for ViT-L16, p ≤ 0.01 for other backbones) with the one-side paired t-test.
T3A outperforms most existing DG algorithms. For example, in VLCS, the proposed method achieves 80.0%, which is significantly better than the prior best-reported score, 78.8%. Similarly, it is 85.3% for PACS, 68.3% (best) for OfficeHome and 47.0% for TerraIncognita, and 70.1% in average (third best). Note that the scores we reproduce are much worse than the reported scores in PACS, which reduces the average performance. In PACS, the improvement from ERM by the proposed method is 1.7 points, which is the most significant improvement from any reported value.
For a more accurate comparison, we also examined whether the proposed method would improve the performance of CORAL, which had achieved the best performance in existing reports. As shown in the third block of Table 1, the proposed method can also improve the performance of CORAL. The average improvement is 0.6 points. Average performance is 70.4%, which is the best among all algorithms. Note that, similar to ERM, the reproduced score and the reported score are slightly different (and the score in PACS is particularly low).
T3A outperforms existing test-time adaptation methods. We further compare T3A with other test-time adaptation methods when backbone networks employ the BN layer so that we can make a fair comparison with the prior BN-based methods [43, 52]. We used ResNet18 w/BN and ResNet50 w/BN as backbone networks. Note that the results of ResNet in Table 1 and Table 2 do not use the BN layer since it is the default option in [17]. Therefore, the results below are not directly comparable to the DG method shown in Table 1 and Table 2 of the current manuscript.
We tested the following six baselines in addition to T3A, Tent-BN, and Tent-C. (1) Tent-Full updates BN statistics and transformations, which is the same as the original proposal [52]. (2) BN-Norm update BN statistics but fixes transformations parameters [43]. (3) PL (Pseudo Label) [26] updates entire networks by minimizing the cross-entropy between prediction and the pseudo label. Following [52], we assign the pseudo label if the predictions are over a threshold (0.9 in our experiment). (4) PL-C updates the linear classifier by minimizing the above-mentioned pseudo-label loss. (5) SHOT [34] updates feature extractor to minimize entropy, diversity regularizer, and pseudo-label loss. While [34] originally proposed SHOT in the context of source-free domain adaptation (offline adaptation setup), the method itself can be transferred to our setup. Note that the original SHOT uses the label-smoothing when training on the source domain. However, we focus on the adaptation method, and therefore the source model is the same as the other baselines for the fair comparison. (6) SHOT-IM [34] updates the feature extractor to minimize entropy and the diversity regularizer.
In summary, (1) all baseline except BN-Norm and T3A use stochastic optimization during test-time, which is not desirable since it may cause catastrophic failure and must increase computational costs. (2) Tent-Full and BN-Norm are powerful yet constrained to be applicable only if the architecture uses BN.
Table 3 compares results under the training-domain model selection as with Table 1 and Table 2. For clarification, we also compare performance under the oracle model selection (28 in Appendix C ), where one can use the validation set on the target domain (20% of all data as described above). We can make the following observations. (1) The proposed method still outperformed all baselines in both backbone networks. Among baselines, only Tent-Full and PL-C perform better than None (w/o adaptation) on average. (2) When we select the hyper-parameters with a test-domain validation set, Tent-Full gives comparable performances with the proposed method. In addition, compared to T3A and PL-C, the T3A performs better under both model selection strategies. These results clarify the difficulty of model selection in optimization-based methods and the merit of the proposed optimization-free approach in this setup. (3) Updating feature extractor (or the large portion of the parameters) does not work well in general, while it is common in SFDA (offline) setup. The results suggest that we need different treatments on online and offline setup.

heading: 5 Discussion and Conclusion
text: This paper presents a new domain generalization algorithm, T3A, which adjusts its predictions during test-time by itself. We show that T3A reduces prediction entropy (Fig. 1-c), and more importantly, generalization error on unseen domain (Table 1). T3A can adapt different domain generalization
algorithms for training-phase (the third block of Table 1), and different backbone networks (Table 2). Note that this property is important in practice because a better backbone network usually give significant performance gains. For example, Table 2 suggest a practitioner should try HViT, which outperforms ResNet50 by a large margin (7.8 points in average) if computational resources allow. T3A can boost HViT’s performance by 1.0 points. Unlike existing studies that update the model with SGD during testing, the proposed method is optimization-free. Therefore, the computational overhead is negligible, and the behavior is unlikely to become unstable, making T3A especially suitable for online settings where the model needs to (adapt and) predict online as with typical DG.
One of the limitations of T3A is how to extend it beyond the classification problem. Since the proposed method creates a template online for each prediction class, it is not trivial to adapt it for continuous prediction. Note that Tent have the same problem, as prediction entropy is hard to compute in the regression case. It is a future task to apply this idea to a broader range of problem settings.
Another potential drawback of the proposed method is that the model can change at any time, making it difficult to thoroughly test its behavior in advance. This may raise ethical concerns in some sensitive applications, making it more difficult to sanitize the model and ensure it does not make unfair decisions. In such a situation, fully online adaptation might be difficult, and one may want to update the model offline. We encourage examination of each of these works on the frontier of test-time adaptation.
Although accuracy was greatly improved, there is massive room for improvement as the performance in the unknown domain is still significantly worse than performance in the known domain. From a probabilistic viewpoint, the templates of each class can be regarded as the statistics of the P (Z|Y ), and our method adjusts it. As it is connected to the prediction P (Y |Z) = P (Z|Y )P (Y )P (Z) , adjusting it can correlates the prediction. From this perspective, using the average templates might be too restrictive, and one can use higher-order statistics to improve performance. Alternatively, one can retain all reliable samples, approximating P (Z|Y ) as empirical distribution. We hope that the findings of this paper will lead to a better test-time adaptation method and lead to the development of machine learning systems that work well in unknown environments.

heading: Acknowledgements
text: This work has been supported by JSPS Grant-in-Aid for Early-Career Scientists Number JP18K18101 and the Mohammed bin Salman Center for Future Science and Technology for Saudi-Japan Vision 2030 at The University of Tokyo (MbSC2030). Computational resource of AI Bridging Cloud Infrastructure (ABCI) provided by National Institute of Advanced Industrial Science and Technology (AIST) was used.

Review summaries:
This paper introduces one algorithm for domain generalization and test-time template adjuster (T3A) for developing a model that performs well under conditions that are different from training conditions. The designed T3A is composed with two procedures: (1) compute a pseudo prototype representation for each class based on unlabeled online dataset, then (2) classify the data based on its distance to the pseudo prototypes. The proposed T3A can improve the performance on the unseen domains, and also outperform the state-of-the-arts.

This paper proposes a new online method for test-time adaptation to domain shift. Specifically, the final layer of a network (i.e. the classifier) is adapted to test data using pseudo-prototypes, which combine the ideas of pseudo-labelling [1] and prototypical representations [2]. That is, pseudo-labels define the support set for each class, and the centroids of these support sets are the prototypes. Experiments demonstrate incremental but reliable performance improvements in the test domain using this optimization-free method.

This paper proposed a test-time adaptation (TTA) solution for domain generalization (DG) problems. It exploits prototypical representations, which are used to classify test samples and are refined over time. The method performs well, when compared with standard DG methods and the recently introduced TTA method "TENT".