Variational Pseudo Labels for Meta Test-time Adaptation
Authors: Sameer Ambekar
Zehao Xiao
Jiayi Shen
Xiantong Zhen
Cees G. M. Snoek
Venue: Submitted to ICLR 2023 (2023)
Keywords: Test-time adaptation
probabilistic framework
variational pseudo label
meta learning

Abstract: Test-time model adaptation has shown great effectiveness in generalizing over domain shifts. A most successful tactic for test-time adaptation conducts further optimization on the target data using the predictions by the source-trained model. However, due to domain shifts, the source-trained model predictions themselves can be largely inaccurate, which results in a model misspecified to the target data and therefore damages their adaptation ability. In this paper, we address test-time adaptation from a probabilistic perspective. We formulate model adaption as a probabilistic inference problem, which incorporates the uncertainty into source model predictions by modeling pseudo labels as distributions. Based on the probabilistic formalism, we propose variational pseudo labels that explore the information of neighboring target samples to improve pseudo labels and achieve a model better specified to target data. By a meta-learning paradigm, we train our model by simulating domain shifts and the test-time adaptation procedure. In doing so, our model learns the ability to generate more accurate pseudo-label distributions and to adapt to new domains. Experiments on three widely used datasets demonstrate the effectiveness of our proposal. 

Under review as a conference paper at ICLR 2023
VARIATIONAL PSEUDO LABELS FOR META TEST-TIME ADAPTATION
Anonymous authors Paper under double-bling review
ABSTRACT
Test-time model adaptation has shown great effectiveness in generalizing over do- main shifts. A most successful tactic for test-time adaptation conducts further optimization on the target data using the predictions by the source-trained model. However, due to domain shifts, the source-trained model predictions themselves can be largely inaccurate, which results in model misspecified to the target data and therefore damages their adaptation ability. In this paper, we address test-time adaptation from probabilistic perspective. We formulate model adaption a probabilistic inference problem, which incorporates the uncertainty into source model predictions by modeling pseudo labels distributions. Based the prob- abilistic formalism, we propose variational pseudo labels that explore the informa- tion of neighboring target samples improve pseudo labels and achieve model better specified to target data. By meta-learning paradigm, we train our model by simulating domain shifts and the test-time adaptation procedure. doing so, our model learns the ability to generate more accurate pseudo-label distributions and to adapt to new domains. Experiments on five widely used datasets demonstrate the effectiveness of our proposal.
INTRODUCTION
Deep neural networks start to exhibit generalizability problems and suffer from performance degra- dation as soon as test data distributions differ from the ones experienced during training. (Geirhos et al., 2018; Recht et al., 2019). To deal with the distribution shift, domain adaptation, e.g., (Saenko al., 2010; Long et al., 2015; Lu et al., 2020; Li al., 2021) and domain generalization, e.g., (Muan- det et al., 2013; Motiian et al., 2017; Li et al., 2017; 2020) have proven effective tactics. However, these two settings either require a large number of (unlabeled) target data during training or do not consider any target information during generalization at all. Both of which are not necessarily valid assumptions in realistic scenarios.
Test-time adaptation, e.g., (Sun et al., 2020; Varsavsky et al., 2020; Wang et al., 2021) goes beyond these two setting and introduces a new learning paradigm, which trains a model on source data and further optimizes it using the unlabeled target data at test time adapt the target domain. One widely applied strategy for test-time adaptation updates model parameters by self-supprvision (Liang et al., 2020; Wang et al., 2021: Iwasawa & Matsuo, 2021; Niu al., 2022). However, due to domain shifts, the source-model predictions on the target samples can be uncertain and inaccurate. As sutf--uperviiion-aseeeee test-time adaptation is often achieved by optimization with pseudo labels or entropy minimization based on the source source-trained model predictions, the model can be overcon- fident on some mispredictions. As result, the adapted model becomes unreliable and misspecified (Wilson & Izmailov, 2020) to the target data.
In this paper we make three contributions. First, we address test-time adaptation a probabilis- tic framework by formulating it as variational inference problem. We define pseudo labels as stochastic variables and estimate a distribution over them by variational inference. By doing so, the uncertainty in source-trained model predictions is incorporated into the adaptation to the target data at test time. Second, thanks to the proposed probabilistic formalism, it is natural and conve- nient to utilize variational distributions to leverage extra information. By hinging this benefit, design the variational pscudo labels to explore the neighboring information of target samples into the inference of the pseudo label distributions. By doing so, the variational pseudo labels are more

---

Under review as a conference paper at ICLR 2023
accurate, which enables the source-trained model to be better specified target data and therefore conducive to model adaptation. Third, we adopt a meta-learning paradigm for optimization to sim- ulate test-time adaptation source domains. More specifically, the model is exposed to domain shifts iteratively and optimized to learn the ability of adapting to unseen domains. We conduct ex- periments three widely-used benchmarks to demonstrate the promise and effectiveness of method for test-time adaptation.
2 METHODOLOGY
2.1 PRELIMINARY
We were given data from different domains defined on the joint space X' x V/ where X' and y denote the data space and label space, respectively. The domains are split into several source domains
(x., yz)² ===1 and target domains D,= (x), Ji=1 The goal is to train model on source domains that is expected to generalize well on the (unseen) target domains.
To this end, test-time adaptation methods, e.g., (Wang et al., 2021; Zhang et al.. 2021; Niu et al., 2022), have recently been proposed. These methods adapt the source-trained model by optimization to target domains at test time.
common strategy in these methods is that the model 0 is first trained on source data D, by minimizing a supervised loss Ltrain (a)inn0)=30)) (0)=EEx.,p) Y')E,, (x,, O)]; and then at test time they adapt the model 0, to the target domain by optimization with certain surrogate losses, e.g., entropy minimization, based on unlabeled test data, which is formulated as:
Liest (0) Ex.ED, LExx: 0.)],
(1)
where the entropy is calculated on the source model predictions. However, test samples from the target domain could be largely misclassified by the source model due to the domain shift, resulting in large uncertainty in the predictions. Moreover, the entropy minimization tends to update the model with high confidence even for the wrong predictions, which would cause a misspecified model for the target domain.
To solve those problems, in this work we address test-time model adaptation from probabilistic perspective. We propose probabilistic inference framework that models the uncertainty of the source-model predictions by defining distributions over pseudo labels. Moreover, under the prob- abilistic formalism, we propose designing variational pseudo labels, which enables the model to incorporate the neighboring information in test samples to combat domain shifts. We adopt a meta- learning paradigm for optimization, which simulates the domain shifts and adaptation procedure. By doing so, the model learns to acquire the ability to further adapt itself with pseudo labels to unseen target domains. We provide graphical illustration to highlight the differences between common test-time adaptation and our proposals in Figure 1.
2.2 PROBABILISTIC TEST-TIME ADAPTATION WITH LATENT PSEUDO LABELS
We first provide probabilistic formulation for test-time adaptation based pseudo labels. Given the target sample x, and the source- trained model 6s, we would like to make predictions on the target sample. To this end, we formulate the predictive likelihood follows:
ply:/xe, 0.) = ply./x,,..//o...l ((), 0.)d0, ~ p(ye/xe, 0,),
where we use the value Or obtained by the maximum a posterior (MAP) to approximate the integra- tion (Finn al., 2018). Intuitively, the MAP approximation is interpreted as inferring the posterior over 0,: p(0, Xx, 0.) = 5(0, 07), which we obtain by adapting 0, using the target data x,-
To model the uncertainty of predictions for more robust test-time adaptation, treat pseudo labels as stochastic variables the probabilistic framework as shown in Figure 1 (b). The pseudo labels obtained from the source model predictions, which follows categorical distributions. Then
2

---

Under review as a conference paper at ICLR 2023
observed
unobserved
observed during training only
Y
X!
Ye
0₂
V
X
Oe
Yr
(a) Original test-time adaptation (Wang et al. 2021)
(b) Test-time adaptation with probabilistic formulation (This paper)
(c) Meta-adaptation with variational pseudo labels (This paper)
Figure 1: Graphical illustrations for test-time adaptation. (a) The original test-time adaptation algorithm (Wang et al., 2021) obtains an adapted 0, by entropy minimization of the unlabeled target data X2 on source trained model 0,. (b) Our probabilistic formulation models the uncertainty of pseudo labels p(ý:) for more robust adaptation. (c) Furthermore, we propose meta adaptation with variational pseudo labels to incorporate neighboring target information into pseudo label generation and train the model under the meta-learning setting. Note that ye are labels of meta-target data and observed only training. The actual labels y, of target data are unavailable for test-time adaptation at inference time.
reformulate eq. (2) as follows:
p(y: 0.) 1 pi)) p(erls,, 0, 0.)p(5)1xx.)) 0.)))
(3)
~ 0, 0;)1,
where 0r is the MAP value of p(0, Vr, x,, 02), which is obtained via gradient descent the data X, and the corresponding pseudo labels starting from 0.. The formulation allows us to sample different pseudo labels from the categorical distribution p(ý:) adapt the model 02, which takes into account the uncertainty of predictions by the -uurce-trained model.
By approximating the expectation of p(ýc) with the argmax function on p(ý.). 0 is obtained by gradient descent based on only a point estimation of the pseudo label ). However, due to domain be always correct. The adaptation then
shifts, the a argmax value of p(y+) is not guaranteed similar to entropy minimization (eq. 1), where the adapted model can achieve high confidence but wrong predictions of some target samples due to domain shifts. For example, consider a toy binary classification task, where the predicted probability is [0.4, 0.6] with the ground- truth label [1, 0j. The pseudo label generated by selecting the maximum probability is [0, 1]. which is inaccurate. Optimization based on these labels would give rise to model misspecified target data, failing to adapt to the target domain.
In contrast, probabilistic formulation allows to sample pseudo labels from the categorical distribution p(ýr Xx, 0,), which incorporates the uncertainty of the pseudo label ý, in a principled way. Again using the above example, the pseudo label sampled from the predicted distribution has a probability of 40% to be the ground-truth label, which leads to the adaptation of the model in the correct direction. Therefore, our formulation achieves better adaptation by accessing accurate pseudo labels from the inaccurate prediction distributions.
2.3 VARIATIONAL PSEUDO LABELS
Under the probabilistic formalism, we derive variational inference of pseudo labels. To train the ability of the model to generate better variational pseudo labels and fully utilize the pseudo label distributions for better adaptation, we adopt the meta-learning paradigm to simulate domain shifts and test-time adaptation procedures (Finn et al., 2017; Dou al., 2019; Xiao et al., 2022). We split the source domains D. into meta-source domains D.r and a meta-target domain D, during training. The meta-target domain is selected randomly in each iteration mimic diverse domain shifts.
3

---

Under review as a conference paper at ICLR 2023
To simulate the test-time adaptation and estimation procedure, we maximize the Log-likelihood of the meta-target samples after model adaptation on the veta-target data:
log plgplyexx..oom 0₂) log / plye|xe, 0.))
p(0 15 x 0x )p(S bxv, 0x)d8r
d0,
log / plyulxe, 07 (p(r xv. 0. )dye
(4)
Epss, x,,,0rr) log Xx,, 0.:)].
where p(Ke X 0z)) denotes the distribution of pseudo labels generated by the meta-source model Og on the meta-target data Xy, Oz is the MAP value of p (0 Yv,, Xes, 0.)) similar to eq. (3), which learned to mimic the test-time adaptation procedure.
Under the meta-learning setting, the actual labels Yr of the meta-target data is accessible since source data fully labeled as shown in Figure (c). We then simulate the test evaluation procedure and further supervise the adapted model 0% on its meta-target predictions by the actual labels.
The maximization of the log likelihood of p(ye' Xx, 0%) realised by cross-entreee cross-entropy loss on the meta- meta-target predictions and the actual meta-target labels. Intuitively, the pseudo-label adapted model supervised to achieve good performance on the adapted data. Thus, learns the ability generate better pseudo labels and achieve better adaptation across domain shifts with these pseudo labels on new unseen domains.
Variational pseudo labels. Moreover, we also propose variational pseudo labels that incorporate information of the neighboring target samples to estimate pseudo label distributions that are more robust against domain shifts. The variational pseudo labels is natural and convenient to deployed un- M
der the probabilistic formulation. Assume that we have a batch of meta-target data X = [x] 1=1' we reformulate eq. (4) as:
log ply 1xe, 0,, xv) =log| plyrize, 0v) 1 p p(0 (yr, xe, Oz (pre, welxe, O2, X )dy d0p
(5)
loog| / [ piyeize, O )lle,, we 0, X ) dw
where 07 is the MAP value of p00:01000 [Rr,, X 02). We introduce the latent variable wy to integrate the information of the neighboring target samples X as shown in Figure 1.
To approximate the true posterior of the joint distribution p(ye, wy), we introduce a variational posterior q&ve, we Xy. Osr Xy, Yr), where Y {yt i-n denotes the actual labels of the meta- target data Xe. To facilitate the estimation of pseudo labels, we set the prior distribution as:
p(ye, Wy (x 02 X p(ýe We, Xy ()))p(( (wel X.,)
where P wr 0z X+) is generated by the features of X together with their output values based on 0,..
Similarly, define the variational posterior distribution as: g(Se, We Xre 0g'; Xv, Yr)=p((+r) Soo/Br//eer. We Xe ) (We W O3', 10s::Xx, Xe, Yv).
(7)
where go (w, 10,, . X Y is obtained by the features of X and the actual labels Y, on 0z.
By introducing eqs. (6) and (7) into (5), we derive the evidence lower bound (ELBO) of the log-
likelihood in eq. (5) as follows:
log Xe, O2 X.) > (gg( ) Episer (w x logg(ye X 02))
|Drllgo(w: Oze, Xe, Yr 1lP (wr Oz,, X.)]-
Rather than directly using the meta-source model 0.,, we estimate the pseudo labels yr from the latent variable Wy. which integrates the features of neighboring target samples. By considering the actual labels Ye, the variational distribution utilizes both the target information and categorical information of the neighboring samples. Thus, the variational posterior models the distribution of different categories in the target domain more reliably and produces more accurate pseudo labels to improve model adaptation.

---

Under review as a conference paper at ICLR 2023
2.4 META TIST-TIME ADAPTATION: TRAINING AND INFERENCE
To mimic domain shifts during training, we split each iteration into meta-source, meta-adaptation, and meta-farget simulate the training stage on source domains, test-time adaptation, and test stage on target data, respectively.
Under the meta-lcarning paradigm, the model iteratively exposed to domain shifts and learns the capability to adapt the meta-source model Or to meta-target MAP 0g with the variational pseudo labels. The parameters in the variational inference model & are jointly optimized in order generate better pseudo labels against domain shifts.
Meta-source. We first train and optimize the model on meta-source domains by minimizing the supervised loss:
A. min R "x,,,,, (Fee, (xx'((X',,
where (X yrr) denotes the input-label pairs of samples on meta-source domains. Og are the model parameters trained on the meta-source data.
Meta-adaptation. Once the meta-source-trained model 0- is obtained, we generate the pseudo labels p(rr/ Xy) of the meta-target data with the variational posterior co(w:-10. Xr, Yr). The test-time adaptation procedure is simulated by obtaining O.::
0 = 02 - A1 (CL Frr;
Yv ~ p(ye Wer,
(10)
where Å denotes the learning rate of the optimization in the meta meta-adaptation stage.
Meta-target. Since our final goal is to obtain good performance on the target data after optimization with pseudo labels, we further mimic the test-time inference on the meta-target domain and supervise the meta-target prediction on 02 by maximizing the log-likkliibe likelihoodd, which equal minimizing:
Cmeta ((xp ( Ep(s, w. (xr(X Yr 07.) [ IDx x. [a (we) [ (w:r)], (11)
where yr denotes the ground truth label of X The parameters O are finally updated by
0 A2 A VoLmeta.
(12)
where A2 denotes the learning rate for the meta-target stage. Note that the loss in eq. (11) is computed on the parameters 8% obtained by eq. (10), while the optimization is performed over the meta-source- trained parameters O.r in eq. (12). Intuitively, the parameters are optimized to learn the ability to handle domain shifts, such that adaptation with variational pseudo labels of data from new domains improves the predictions on the new domain.
The parameters in the variational inference model & are jointly trained with 0. To guarantee that the variational pseudo labels do extract the neighboring information for discrimination, we add cross-entropy loss Lie on the variational pseudo labels and the corresponding actual labels during training. Thus, updated by:
- >a - VoLCeeta),
(13)
where A denotes the learning rates.
Test-time adaptation and prediction. At test time, the model trained the source domains with the above meta-learning strategy 0, is adapted by further optimization using eq. (10).
The adapted model is then evaluated on the (unseen) target data D,. The prediction is formulated
p(ye|xe, Oz, X.) f plyxlxee 0.) /
Xr, O.),( wr|xe, Os,, Xx)dy,dwyy dO,
(14)
Bpp(ws)) Bppss//... [log |x,, 0,)],
where 0 is the MAP value of p(0 (x, r., O.). Pel (w.) (w. X,) is generated by the features of X, according to the outputs or common pseudo labels based on 0,.
3 RELATED WORK
Test-time adaptation. By combining the advantages of both domain adaptation (Ganin & Lempit- sky, 2015; Long et al., 2015; Hoffman al., 2018; Lu al., 2020; Tzeng et al., 2017; Shen et al.,
5

---

Under review as a conference paper at ICLR 2023
2022) and domain generalization (Muandet et al., 2013; Li et al., 2017; 2019; Du et al., 2020; Zhou et al.. 2020; 2022), test-time adaptation (Sun et al., 2020; Dubey et al., 2021; Wang et al., 2021; Zhou & Levine. 2021; Chen et al., 2022) and source-free free adaptation (Liang et al., 2020: Eastwood et al., 2021) are proposed to train a model only on source domains while adapting it to unlaheled target data at test time. Several methods update normalization statistics of the model handle do- main shifts (Schneider et al., 2020; Du et al., 2021; Hu et al., 2021). Sun et al. (2020) proposed to fine-tune the model parameters by a self-supervised loss test time. Liu al. (2021) further enhanced the method by introducing a test-time feature alignment strategy. Instead of using an extra self- supe-vuueee loss, Wang et al. (2021) proposed fully test-time adaptation by entropy minimization, which is followed in several recent works (Zhang et al., 2021; Niu et al., 2022; Jang & Chung, 2022).
Zhang et al. (2021) minimized the entropy of the marginal output distribution averaged over multiple augmentations of a single target sample. Niu et al. (2022) proposed an efficient test-time adaptation method without forgetting by adapting with low entropy samples with Fisher regularization. Differ- ent from these methods, Iwasawa & Matsuo (2021) tried to adjust the classifier with pseudo labels of the target data, without fine-tuning the model parameters. Different from these methods, we propose probabilistic formulation of test-time adaptation, which models the uncertainty of pseudo labels for better adaptation. We also introduce variational pseudo labels with meta-adaptation to further learn the ability to improve the pseudo labels and adaptations.
Meta-learning-baset methods (Alet et al., 2021; Xiao et al., 2022; Goyal et al., 2022) have been studied for test-time adaptation before. Alet et al. (2021) learned to adapt with a contrastive loss. Goyal et al. (2022) meta-learned the loss functions of the test-time adaptation for better adaptation. Xiao et al. (2022) proposed single sample generalization, which adapts the model to each individual target sample through mimicking domain shifts during training. Our method also learns the adaptation ability under the meta-learning setting. We design meta-adaptation to simulate the test-time adaptation procedure and supervise it based on our probabilistic formulation. We further supervise the meta-adapted model the meta-target stage to learn the variational pseudo label generation and adaptation ability.
Meta-lcarning.
Pseudo label learning. In pseudo label learning, the motive is to use a model's s best predictions, c.g., high confidence predictions, and use the corresponding samples and their predictions for retraining the model on a given downstream task. Tasks examples include classification (Yalniz et al., 2019; Xie et al., 2020), segmentation (Zou et al., 2020), and object detection (Li et al., 2022). Pham et al. (2021) addresses the problem of confirmation bias in pseudo labeling and utilize a teacher-studen network for the task of image classification. (Zou et al., 2019) utilize the output logits of softmax as the prediction probability and train the model to directly maximize the logits. (Wang et al., 2022) utilizes confidence to approximate the domain difference and applies augmentations to improve quality if below the threshold to update the student model. (Rizve et al., 2021) assumes that there is access to target labels in a supervisd learning and relies on prediction uncertainties leveraged through the labeled samples in target to and generates new pseudo labels. For semi-supervised learning, (Miyato et al., 2018) proposes a regularization method that uses target data without labels. Shu et al. (2018) constructs new source domain using pseudo labelling using available target data during source training. For domain generalization and offline adaptation, Abdo et al. (2009) utilize pseudo labels from network extracting features and depth features. For online adaptation, Chen et al. (2022) utilize a contrastive method and utilize pseudo labels to ensure that same class negative samples are not used in contrastive loss optimization. For semi-supervised adaptation Zhou et al. (2021) utilize pseudo labels and use confidence as a criteria. In contrast to existing works, our method uses meta learning and is probabilistic in nature for test-time adaptation.
4 EXPERIMENTS
4.1 SETTINGS
Five datasets. We demonstrate the effectiveness of our method on image classification and domain generalization settings. We evaluate our method on five widely used datasets in domain general- ization. PACS (Li et al., 2017) consists of 7 classes and 4 domains: Photo, Art painting, Cartoon, and Sketch with 9991 samples. VLCS (Fang et al., 2013) consists of 5 classes and 4 domains: Pas- cal, LabelMe, Caltech, SUN with 10,729 samples. Terralncognita (Beery et al., 2018) consists of 10 classes and 4 domains: Location 100, Location 38, Location 43 and Location 46 with 24,778

---

Under review as a conference paper at ICLR 2023
Table 1: Benefits of our probabilistic test-time adaptation. The experiments are conducted on PACS based on ResNet-18. Our probabilistic formulation achieves better performance than the common test-time adaptation in both online and offline settings. The variational pseudo labels with meta adaptation further improve the overall performance.
Method
Photo
Art-painting
Cartoon
Sketch
Mean
Online Adaptation
Test-time adaptation
93.91
78.52
78,33
74.37
81.28
Our probabilistic test-time adaptation
94.55
80.07
79.14
74.29
82.01
Our method
95.50 #0.2
82.90 =0.4
81.28 +0.5
74.11 +0.7
83.45 +0.2
Offline Adaptation
Test-time adaptation
94.61
80.13
79.39
74.96
82.27
Our probabilistic test-time adaptation
95.15
81.15
80.22
75.42
82.99
Our method
95.80 +.05
84.32
#0.8
83.44 +0.4
74.57 #0.2
84.78
+:036
samples. We follow the training and validation split in (Li et al., 2017) and evaluate the model cording to the "leave-one-out" protocol (Li et al., 2019; Carlucci et al., 2019). We also evaluate our method on the Rotated MNIST and Fashion-MNIST datasets following Piratla et al. (2020), where the images are rotated by different angles as different domains. We use the subsets with rotation angles from 15a to 75 in intervals of 15 as five source domains, and images rotated by 0' and 90 as the target domains.
Two adaptation settings. To demonstrate the effectivvness of our method, we evaluate on both offtine and online test-time adaptation settings. Offline test-time adaptation assumes there are al- ready unlabeled target data available for adaptation. To be close to real-world application where is difficult to access the whole target set, we use different amounts of target data for offline adapta- tion. We adapt the model on the available target data and evaluate it on the entire target set without continuously fine-tuning on the entire target set. We also evaluate our method for online test-time adaptation (Iwasawa Matsuo, 2021). real-world application, as aforementioned. instead of ac- cessing the entire target set, usually obtain unlabeled target data in online manner. To achieve continuous adaptation and improvement of the model on target data, we increment the target data iteratively and keep adapting and evaluating the model on the online target data.
Implementation details We make use of ResNet-18 for all our experiments and ablation studies and report the accuracies on ResNet-50 for comparison as well. The backbones are pretrained on ImageNet same as the previous methods. During training, we use a varied learning rate throughout the model. We set the learning rate for the pretrained ResNet to 5e-5 and the learning rate of the variational module and classifiers as le-4 for all datasets. During test-time adaptation, we utilize a learning rate of 1e-4 for all layers. We intend release the code acceptance of this paper.
4.2 ABLATION STUDIES
Benefits of our probabilistic test-time adaptation We first investigate the effectiveness of our probabilistic formulation of test-time adaptation and its meta-learned variational pseudo labels. To demonstrate the benefits of the probabilistic formulation, we conduct test-time adaptation with eq. (3) and compare it with common test-timo adaptation tactic (Wang et al., 2021), as in eq. (1). As shown in Table 1, both adaptation with and without probabilistic formulation achieve good im- provements over the ERM baseline. Our probabilistic test-time adaptation performs better than the common one on most target domains for both the online and offline adaptation settings, which demonstrates the benefits of modeling uncertainty during adaptation at test time.
Moreover, we incorporate the distribution pseudo labels into the probabilistic formulation, and further propose the variational pseudo labels with meta-adaptation based the pseudo label dis- tributions. As shown in the fourth and last row in Table 1, our method further improves the per- formance of both online and offline adaptation, demonstrating the effectiveness of the variational pseudo labels with meta- adttaaadttttioo... With the probabilistic formulation, it is natural and simple to define the problem variational inference problem and solve the problem under the meta-learning

---

Under review as a conference paper at ICLR 2023
86
Variational Pseudo Labels
84
Accuracy
84
Accuracy
83
82
82
81
Variational Pseudo Labels
Pseudo Labels
80
80
50%
75%
100%
20
40
60
Target data for adaptation
Iterations
Figure 2: Benefits variational pseudo labels. The experiments are conducted on PACS with ResNet-18 18 as the backbone. We compare our variational pseudo labels with common pseudo labels with different amounts of adaptation data under the offline settings (left figure). Our variational pseudo labels achieve better overall accuracy consistently. We also provide the accuracy along with adaptation steps on art-painting the right figure. Our method adapts faster and achieves better performance than using the common pseudo labels.
Table 2: Ablation of meta-learning setting:
We conduct the below experiments on PACS using ResNet- 18. In the first framework, do not meta-learning across all stages. We observe that our method performs better on all domains comparatively.
Settings
Photo
Art-painting
Cartoon
Sketch
Mean
W/o meta-learning
94.76 =1.0 80.7 20.5
78.87 207 68.39 =05
80.68
W/ meta-learning
95.80 =0. 84.32 1..8
83.44 20. 74.57 =02
84.78 #0.36
framework. The results further demonstrate the benefits and importance of our probabilistic formu- lations of test-time adaptation.
Benefits of variational pseudo-labels Based the probabilistic formulation, introduce vari- ational pseudo labels to incorporate the target information of the neighboring target data for each target sample. To demonstrate the effectiveness of our variational pseudo label, compare it with the normal pseudo labels drawn directly from the prediction distributions of source-trained models. We evaluate the methods in offline adaptation settings with different amounts of target data. As shown in Figure 2 (left), adaptation with our variational pseudo labels achieves better overall results than the normal pscudo labels consistently. We also provide the adaptation results along with adap- tation steps in Figure 2 (right). Starting from the same baseline accuracy, the variational pscudo labels achieve faster adaptation than the normal pseudo labels. Adaptation with variational pseudo labels is less prone to saturating in performance, leading to better final accuracy.
Benefits meta learning We also investigate the importance of meta-learning in our method. For this experiment, do not make meta-learning for source training and test-time adaptation and only use variational labels that have been generated for adaptation. We observe that meta-learning indeed helps in formulation. Without meta-learning, it is difficult for the model to learn the ability to handle domain shifts. Thus, there a significant decrease in accuracy as shown in Table 2.
Offline adaptation with less target data In real applications, it is difficult to access the entire target set at once for adaptation. Therefore, we estimate our method under the offline test-time adaptation settings with different amounts of target data. As shown in Figure 3, the accuracy in- creases obviously with small numbers of target samples, e.g., 10% and 25%. However, the overall accuracy tends to saturate when keeping increasing the number of target data for adaptation. This indicates that our method is able to achieve good adaptation under the offline adaptation setting with small amounts of target data, showing the applicability of the proposed method in practice.
8

---

Under review as a conference paper at ICLR 2023
Baseline Adaptation with 10% data Adaptation with 25% data
Adaptation with 50% data
Adaptation with 75% data
Adaptation with 100% data
Accuracy
85
70
Photo
Art-painting
Cartoon
Sketch
Mean
Target domains
Figure 3: Offine adaptation with less target data. The experiments are conducted on PACS using ResNet-18 averaged over five runs. Under offline-adaptation setting we observe that with incre- ments in the amount of test data samples the accuracy increases steadily for individual domains.
Table 3: Comparisons on common DG datasets. The experiments are conducted on all datasets averaged over five runs. We provide the results of our method for the online setting. Our method also performs better than the state-of-the-ard domain generalization methods across all datasets.
Algorithm
PACS
VLCS
Terralncognita
ResNet-18 18
ResNet-50
ResNet-18
ResNet-18
ERM
79.29
83.21
74.88
40.62
Standard DG Methods
MASF (Dou et al., 2019)
81.0
82.7
ER (Zhao et al., 2020)
81.5
85.3
74.4
Test-time Adaptation Methods
Tent-By BN (Wang et al., 2021)
81.3
83.7
61.3
39.8
SHOT (Liang et al., 2020)
82.4
84.1
65.2
33.5
T3A (lwasawa & Matsuo, 2021)
81.7
84.5
76.5
41.6
TAST (Jang & Chung, 2022)
81.9
84.1
77.3
42.6
Our Method
83.4 10.4
85.5 10.3
77.8
46.2 10.4
4.3 COMPARISONS
To further demonstrate the effectiveness of our method, we compare our method with some state- of-the-art test-time adaptation methods and standard domain generalization methods. Table shows the results on PACS, VLCS and TerraIncognit. using ResNet-18. Compared with the other state-of- the-art domain generalization methods and test-time adaptation methods with the online adaptation setting, method performs better. On datasets such PACS and Terralncognita, state-of-the-arr methods significantly. We also report results on the PACS datasct using ResNet-50 the backbone, the performance of our method is competitive and better than most of the state-of- the-art methods. We provide the detailed comparison in Appendix
better by
5 CONCLUSION
We propose to cast test-time adaptation as a probabilistic inference problem and model pseudo- labels as distributions in the formulation. By modeling the uncertainty into the pseudo label distribu- tions, the probabilistic formulation mitigates adaptation with inaccurate pseudo labels or predictions, which arises due domain shifts and lead to misspecified models after adaptation. Based on the probabilistic formulation, we further propose variational pseudo labels under the meta-adaptation paradigm, which exposes the model to domain shifts and learns the ability adapt with pseudo la- bels incorporating target information of the neighboring target samples. Ablation studies and further comparisons show the effectiveness of our method on five common domain generalization datasets.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Aous A Abdo, Markus Ackermann, Marco Ajello, WB Atwood, Magnus Axelsson, Luca Baldini, Jean Ballet, Guido Barbiellini, Denis Bastieri, Milan Battelino, al. Measurement of the cosmic ray c++ e- spectrum from 20 gev to tev with the fermi large area telescope. Physical Review Letters, 101118))18110, 2009.
Ferran Alet, Maria Bauza, Kenji Kawaguchi, Nurullah Ciray Kuru, Tomás Lozano-Pérez, and Leslie Kaelbling. Tailoring: encoding inductive biases by optimizing unsupervised objectives at predic- tion time. Advances in Neural Information Processing Systems, 34:29206-2999 2021.
Sara Beery, Grant Van Hor,, and Pietro Perona. Recognition in terra incognita. In Proceedings of the European conference computer vision (ECCV). pp. 456-4744 066-473,22 2018.
Fabio M Carlucci, Antonio Innocente, Silvia Bucci, Barbara Caputo, and Tatiana Tommasi. Do- main generalization by solving jigsaw puzzles. In IEEE Conference Computer Vision and Putlern Recognition, pp. 2229-2238, 2019.
Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 295-305, 2022.
Qi Dou, Daniel C Castro, Konstantinos Kamnitsas, and Ben Glocker. Domain generalization via model-agnostic learning of semantic features. In Advances in Neural Information Processing Systems, 2019.
Yingjun Du, Jun Xu, Huan Xiong, Qiang Qiu, Xiantong Zhen, Cees M Snoek, and Ling Shao. Learning to learn with variational information bottleneck for domain generalization. In European Conference on Computer Vision, pp. 200-216, 2020.
Yingjun Du, Xiantong Zhen, Ling Shao, and Cees G M Snock. MetaNorm: Learning to normal- ize few-shot batches across domains. International Conference on Learning Representations, 2021.
Abhimanyu Dubey, Vignesh Ramanathan, Alex Pentland, and Dhruv Mahajan. Adaptive meth- ods for real-world domain generalization. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 14340-14349, 2021.
Cian Eastwood, Ian Mason, Christopher KI Williams, and Bernhard Schölkopf. Source-free adap- tation to measurement shift via bottom-up feature restoration. arXiv preprint arXiv:2107.05446, 2021.
Yuming Fang, Weisi Lin, Zhenzhong Chen, Chia-Ming Tsai, and Chia-Wen Lin. A video saliency detection model in compressed domain. IEEE transactions on circuits and systems for video technology, 24(1):27-38, 2013.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, pp. 1126-113 1135. PMLR, 2017.
Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic model-agnostic meta-learning. Advances in neural information processing systems, 31, 2018.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pp. 1180-1189. PMLR, 2015.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis. Matthias Bethge. Felix A Wichmann, and Wieland Brendel. Imagenet- trained cnns are biased towards texture; increasing shape bias im- proves accuracy and robustness. arXiv preprint arXiv:1811.12231 2018.
Sachin Goyal, Mingjie Sun, Aditi Raghunathan, and Zico Kolter. Test-time adaptation via conjugate pseudo-labels. arXiv preprint arXiv:2207.096400 2022.
10

---

Under review as a conference paper at ICLR 2023
Ishaan Guirajani and David Lopez-Paz. In search of lost domain generalization. In International Conference on Learning Representations, 2020.
Judy Hoffman, Eric Tzeng, Taesung Park, Jun- Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell. Cycada: Cycle-consistent adversarial domain adaptation. In International Conference Machine Learning, pp. 1989-1998. PMLR, 2018.
Xuefeng Hu, Gokhan Uzunbas, Sirius Chen, Rui Wang, Ashish Shah, Ram Nevatia, and Ser-Nam Lim. Mixnorm: Test-time adaptation through online normalization estimation. arXiv preprint arXiv:21 2021.
Yusuke Iwasawa and Yutaka Matsuo. Test-time classifier adjustment module for model-agnostic domain generalization. Advances in Neural Information Processing Systems, volume 34, 2021.
Minguk Jang and Sac-Young Chung. Test-time adaptation via self-training with nearest neighbor information. arXiv preprint arXiv:2207 10792, 2022.
Bo Li, Yezhen Wang. Shanghang Zhang, Dongsheng Li, Kurt Keutzer, Trevor Darrell, and Han Zhao. Learning invariant representations and risks for semi-supervised domain adaptation. In IEEE Conference Computer Vision and Pattern Recognition, pp. 1113, 2021.
Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generalization. In IEEE International Conference on Computer Vision, pp. 5542-5550, 2017.
Da Li, Jianshu Zhang, Yongxin Yang, Cong Liu, Yi-Zhe Song, and Timothy M Hospedales. Episodic training for domain generalization. In JEEE International Conference on Computer Vision, pp- 1446-1455, 2019.
Haoliang Li, YuFei Wang, Renjie Wan, Shiqi Wang, Tie-Qiang Li, and Alex Kot. Domain gener- alization for medical imaging classification with dinear-depyndency regularization. arXiv preprint arXiv:2009. 2020.
Hengduo Li, Zuxuan Wu, Abhinav Shrivastava, and Larry S Davis. Rethinking pseudo labels for semi-supervised object detection. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 36, pp. 1314-1322, 2022.
Jian Liang, Dapeng Hu, and Jiashi Feng. Do really need access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning, pp. 6028-6039. PMLR, 2020.
Yuejiang Liu, Parth Kothari, Bastien van Delft, Baptiste Bellot-Gurlet, Taylor Mordan, and Alexan- dre Alahi. Tit++: When does self-supervised test-time training fail or thrive? In Advances in Neural Information Processing Systems, volume 34, 2021.
Mingsheng Long, Yue Cao, Jiammin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. In International Conference on Machine Learning, pp. 97-105. PMLR, 2015.
Zhihe Lu, Yongxin Yang, Xiatian Zhu, Cong Liu, Yi-Zhe Song, and Tao Xiang. Stochastic classi- fiers for unsupervised domain adaptation. IEEE Conference on Computer Vision and Pattern Recognition, pp. 9111 9111-9120, 2020.
Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. Virtual adversarial training: a regularization method for supervised and supervise learning. IEEE transactions pattern analysis and machine intelligence, 41(8):1979-199 1993, 2018
Saeid Motiian, Marco Piccirilli, Donald A Adjeroh, and Gianfranco Doretto. Unified deep super- vised domain adaptation and generalization. In IEEE International Conference on Computer Vision, pp. 5715-5725, 2017.
Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via invariant feature representation. International Conference on Machine Learning, pp. 10-18. PMLR, 2013.
11

---

Under review as a conference paper at ICLR 2023
Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. arXiv preprint arXiv:2204.02610, 2022.
Hieu Pham, Zihang Dai, Qizhe Xie, and Quoe V Le. Meta pseudo labels. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 11557-11568, 2021.
Vihari Piratla, Praneeth Netrapalli, and Sunita Sarawagi. Efficient domain generalization via common-specific low-rank decomposition. In International Conference on Machine Learning, pp- 7728-7738. PMLR, 2020.
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? In International Conference on Machine Learning, pp. 5389-5400.
PMLR, 2019.
Mamshad Nayeem Rizve, Kevin Duarte, Yogesh S Rawat, and Mubarak Shah. In defense of pseudo- labeling: An uncertainty-awant pseudo-label selection framework for semi-supervised learning. arXiv preprint arXiv:2101.06322 2021.
Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In European Conference on Computer Vision, pp. 213-226, 2010.
Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. Ad- vances in Neural Information Processing Systems, 33:1 13:11539-11555 2020.
Kendrick Shen, Robbie M Jones, Ananya Kumar, Sang Michael Xie, Jeff Z HaoChen, Tengyu Ma, and Percy Liang. Connect, not collapse: Explaining contrastive learning for unsupervised domain adaptation. In International Conference on Machine Learning, pp. 19847-19878. PMLR, 2022.
Rui Shu, Hung H Bui, Hirokazu Narui, and Stefano Ermon. A dirt-t approach to unsupervised domain adaptation. arXiv preprint 1802v:1802.08777 08735, 2018.
Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time train- ing with self-supervisin for generalization under distribution shifts. In International Conference on Machine Learning, pp. 9229-999999 9248. PMLR, 2020.
Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 7167-7176, 2017.
Thomas Varsavsky, Mauricio Orbes-Arteaga, Carole H Sudre, Mark S Graham, Parashkev Nachev, and M Jorge Cardoso. Test-time unsupervised domain adaptation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 428-436. Springer, 2020.
Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. International Conference on Learning Repre- sentations, 2021.
Qin Wang, Olga Fink, Lue Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 7201-7211, 2022.
Andrew G Wilson and Pavel Izmailov. Bayesian deep learning and probabilistic perspective of generalization. Advances in neural information processing systems, 13:4699-47008, 2020.
Zehao Xiao, Xiantong Zhen, Ling Shao, and Cees G M Snoek. Learning to generalize across domains on single test samples. In International Conference on Learning Representations, 2022.
Qizhe Xie, Minh- Thang Luong, Eduard Hovy, and Quoc V Le. Self-training with noisy student improves imagenet classification. Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pp. 10687-10698, 2020.
12

---

Under review as a conference paper at ICLR 2023
I Zeki Yalniz, Hervé Jégou, Kan Chen, Manohar Paluri, and Dhruv Mahajan. Billion-scee scale semi-
supervised learning for image classification. arXiv preprint arXiv: 2019.
Marvin Zhang, Sergey Levine, and Chelsea Finn. Memo: Test time robustness via adaptation and augmentation. arXiv preprint arXiv:2110.09506, 2021.
Shanshan Zhao, Mingming Gong, Tongliang Liu, Huan Fu, and Dacheng Tao. Domain general- ization via entropy regularization. In Advances in Neural Information Processing Systems, vol- ume 33, 2020.
Aurick Zhou and Sergey Levine. Training on test data with bayesian adaptation for covariate shift. In Advances in Neural Information Processing Systems, 2021.
Kaiyang Zhou, Yongxin Yang. Yu Qiao, and Tao Xiang. Domain generalization with mixstyle. In International Conference on Learning Representations, 2020.
Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xiang. Mixstyle neural networks for domain generalization and adaptation. arXiv preprint arXiv:2 2021.
Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. IEEE Transactions Pattern Analysis and Machine Intelligence, 2022.
Yang Zou, Zhiding Yu, Xiaofeng Liu, BVK Kumar, and Jinsong Wang- Confidence regularized self-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5982-5991, 2019.
Yuliang Zou, Zizhao Zhang, Han Zhang, Chun-Liang Li, Xiao Bian, Jia-Bin Huang, and Tomas Pfister. Pseudoseg: Designing pseudo labels for semantic segmentation. arXiv preprint arXiv:2010.09713, 2020.
13

---

Under review as a conference paper at ICLR 2023
A DETAILED FORMULATION
We start the objective function from Par,Xy). Here we provide the detailed generating process of the formulation:
p(ye Xr, O2, Xx) = f plyelxe, Or)p(Or(xx Xx,, O2r, d00e
(15)
We then introduce the pseudo labels Vr as the latent variable into eq. (15) and derive as:
p(ye Xe,, 0,,, X²) - plye x Bv)) / p(orlSe, xe, Re, 0g )p(ee) X Ogr, X )dy e
(16)
Theoretically, the distribution p(0r) obtained by p(Ox yy yyx,ee X 0z,) p(y, Xt, Oc) 0g O.). where p(0, ( is the prior distribution. To simplify the formulation, we approximate the inte-
gration of p(0 ) by the maximum posterior (MAP) value of o. We obtain the MAP value by
training the model 0 with inputs X," and pseudo labels ye starting from Oz. The formulation then derived as:
p(yvlxe,,0 0,)) - /
/1 O2 De.Ke)d55 dÿrr.
(17)
To obtain better pseudo labels ÿv,, we further introduce the latent variable wy into eq. (17) and variational posterior of the joint distribution qfe,, wy). The formulation then derived as:
plyel Xx, X.)
f /J plyeixe, 0%) / plye, wrlxe, 02. X dww
(18)
f plye plyvlxx, 07) / g(Ver we xv. O2, Xe, Yv) p(ye, we (xx, Oz X,,) where q(ve, we xr. 0g. Xv, Yv) and p(Ser we Xy,, ,gr, Xx) denote the prior and posterior dis- tributions, respectively.
= - ]
p(Se, , We Xx, Ow, X (d
B IMPLEMENTATION DETAILS
Our train setup follows Iwasawa Matsuo (2021). We use batch size of 70 and train our method using the ERM algorithm Guirajani & Lopez-Paz (2020). As stated, our backbones such as ResNet- 18 and ResNet-34 are pretrained on ImageNet same as the previous methods. During our training, the model with highest validation accuracy is selected for adaptation on the target domain. We use similar settings for all Domain Generalization benchmarks that have been reported the paper. We train all our models on NVIDIA Tesla 1 1080Ti GPU. In Table 8 we report the runtime required for test-time adaptation. We observe that our method is comparable methods that are not optimization free such SHOT and PL. We also in Table report the amount of time consumed during source training.
C DETAILED EXPERIMENTAL RESULTS
Detailed experimental results. In Table 4 we report our detailed performance and comparison to existing methods on PACS with both ResNet-18 and ResNet-50 as the backbone. We observe that method shows an improvement in accuracy compared to other methods especially on "Art- Painting". "Cartoon" and "Photo" Domains. The results show the benefits using Variational Pseudo labels for adaptation.
We also conduct experiments on rotated MNIST and rotated Fashion-MNIST for comparison as shown in Table 5. We follow the settings in Piratla et al. (2020) and use ResNet- 18 the backbone. The conclusion similar to that in PACS. Our method achieves better performance than both the non-adaptive domain generalization methods and adaptation methods.
14

---

Under review as a conference paper at ICLR 2023
Comparisons PACS. The experiments conducted on PACS averaged over five runs. We provide the results of our method under both the online and offiin adaptation settings. Our method better than other methods both settings with different backbones. Our method also performs better than the state-of-the-an domain generalization methods.
Table 4:
Algorithm
Photo
Art-painting
Cartoon
Sketch
Mean
R8sNet-18
Online adaptation
Tent-BN (Wang al., 2021)
93.9 10.3
78.5
78.3
104
74.4
81.3
10.3
T3A (Iwasawa & Matsuo, 2021)
94.7 #0.5
80.4 #07
75.2 =04
76.5
=02
81.7 #04
TAST (Jang & Chung. 2022)
96.4 10.
80.6 105
78.3
110
72.5
108
81.9
104
SHOT (Liang al., 2020)
96.2 +0.3
81.1 +09
79.7 =09
72.5
82.4
20.6
Our Method
95.5 +0.2
82.9 +0.4
81.3 =0.5
74.1
#0.7
83.4
40.2
Offline adaptation
Tent-Bi (Wang al., 2021)
94.6 +:02
80.1 #0.9
79.4 =0.8
75.0
1
82.3
20.3
Our Method
95.8 +:05
84.3 +0.8
83.4 10.4
74.6
#02
84.8
20.4
ResNet-50
Online adaptation
Tent-BN (Wang et al., 2021)
96.0 10.4
84.9 10.
79.8 10.6
75.7
20.7
84.1
20.3
T3A (Iwasawa & Matsuo. 2021)
96.4 +02
86.0 #06
80.3 =09
75.2
#15
84.5
40.3
SHOT (Liang et al., 2020)
96.5 10.5
84.6 #1.7
80.1 =1.3
74.8
22,9
84.
11.0
TAST (Jang & Chung, 2022)
96.9 +0.5
83.8 #0.5
79.1 =0.5
76.4
=0.5
84.1
+0.5
Our Method
96.9 +02
87.0 +0.4
83.8 =0.3
74.3
=0.2
85.5
#0.3
Offiine adaptation
Tent-BN (Wang al., 2021)
96.5 +0.2
85.8 +0.3
81.5 =0.7
76.9
85,1
+0.3
Our Method
97.6 10.2
87.3 104
84.2
102
75.3
10.3
86.1
10.2
Table 5: Comparison rotated MNIST and Fashion-MNIST. The models evaluated on the test sets of MNIST and Fashion-MNIST with rotation angles of 0" and 90%. Our method performs better than both non-adaptive domain generalization methods (Dou et al., 2019; Piratla et al., 2020) and adaptive methods (Wang et al., 2021; Xiao et al., 2022).
MNIST
Fashion
Dou et al. (2019)
93.2
72.4
Piratla al. (2020)
94.7
78.0
Wang et al. (2021)
95.3
78.9
Xiao al. (2022)
95.8
80.8
Our method
95.9 +0.1
82.4
+002
Extra ablations We also provide extra ablation studies on the pseudo label generation and adap- tation with our variational pseudo labels. We directly make predictions using the variational pseudo labels that are generated by sampling from the pseudo label distributions at test time. As shown in Table 6, the predictions based on the pseudo-label distributions are better than the ERM baseline, demonstrating that our variational pseudo labels are better than the original pseudo labels. More- over, after adapting the model parameters by our variational pseudo labels, the performance further improves obviously.
Table 6: Investigating by evaluating directly using the pseudo label distributions. We conduct the below experiments on PACS using ResNet-18, Our method that adapts the models with varia- tional pseudo labels performs better than the prediction by directly sampling from the pseudo label distributions.
Settings
Photo
Art-painting
Cartoon
Sketch
Mean
Prediction by pseudo label distributions
95.97
81.23
79.19 207
73.22 10.5
82.40 40.55
Our method
95.80 40.5
84.32 208
83.44 204
74.57 102
84.78 40.36
15

---

Under review as a conference paper at ICLR 2023
Table 7: Runtime required for source training on PACS using ResNet-18 a backbone net- work. The proposed method has larger time costs during training due to the meta-learning strategy but introduces few extra parameters.
Parameters
Time for 10000 iterations
ERM Our method
11.18M
6.5 hours
1 11.96M
14.6 hours
Table 8: Runtime averaged for datasets using ResNet-18 as backbone network. The proposed
method has similar or even better time costs at test time with the other test-time adaptation methods.
VLCS
PACS
Terra-Incognita
Tent (Wang et al., 2021)
7m 28s
3m 16s
10m 34s
Tent-BN (Wang et al., 2021)
2m 8s
33s
2m 58s
SHOT (Liang et al., 2020)
8m 9s
4m 22s
12m 40s
T3A (Iwasawa & Matsuo, 2021)
2m 9s
33s
2m 59s
TAST (Jang & Chung, 2022)
10m 34s
9m 30s
26m 14s
Our method
2m 20s
5m 33s
14m 30s
Time cost analyses. We also provide the time cost of our method in both the training (Table 7) and inference stage (Table 8). As we utilize the meta-learning strategy learn the ability handle domain shifts during training, the time cost during training is larger than the ERM baseline. More- over, compared with the ERM baseline, our variational pseudo-label learning and meta-learning framework only introduce few parameters. Since the meta-learning strategy only complicates the training process, our method has similar runtime in inference compared with the other test-time adaptation methods, e.g., Tent (Wang et al., 2021) and source-free domain adaptation methods, e.g., SHOT (Liang et al., 2020).
16


Reviews:
[{"review_id": "S8JSpHhBwi", "reviewer": "Reviewer_vwGZ", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Test-time adaptation updates a model during testing to improve its generalization to different data.\nA popular choice of test-time optimization is entropy minimization or pseudo-labeling, which trains on the model's own predictions.\nThis work seeks to prepare a model for such test-time optimization by meta-learning (1) the model parameters for a better initialization and (2) the pseudo-labeling inference to provide better targets for the loss.\nAs a first step, a probabilistic alternative to deterministic pseudo-labels is proposed, and then a variational distribution is defined to condition on the test inputs and their relationships.\nThe probabilistic pseudo-labels simply sample new pseudo-labels given the pseudo-label probabilities.\nThe variational distribution for pseudo-labels conditions on the given input and the other samples in the batch through a latent weighting $w$.\nTraining has three phases: meta-source optimization learns the source model parameters, meta-adaptation infers the variational pseudo-labels and optimizes the target model parameters, and then meta-target optimization updates the source model parameters by the meta-loss to achieve better meta-adaptation optimization.\nAt the same time, the variational pseudo-labeling parameters are jointly updated alongside the model parameters, according to the meta-loss and a cross-entropy loss between the pseudo-labels and true labels.\nTesting proceeds as usual for test-time adaptation, with alternating predictions and updates, as shown by Eqs. 10 and 14.\nThat is, given the meta-learned model and pseudo-labeling inference, the model parameters are updated during testing according to the cross-entropy loss on the predicted pseudo-labels.\nExperiments evaluate adaptation on PACS, a common domain generalization dataset, along with toy datasets based on MNIST with artificial domains made by rotating images, while comparing with recent test-time adaptation methods.\nAccuracy improves by 1-2 points with the proposed probabilistic pseudo-labels and variational pseudo-labels (Table 1, Table 3, Table 4).\nAnalysis experiments show that the variational pseudo-labels reliably improve a little bit on standard pseudo-labels: overall (Figure 2, left), across iterations (Figure 2, right), and across different amounts of data for adaptation (Figure 3).", "summary_of_the_review": "This work brings a probabilistic perspective to pseudo-labeling and contributes a meta-learning scheme for training the model parameters and a pseudo-label predictor.\nThe proposed stochastic and variational pseudo-labels do seem to improve marginally on PACS, but the improvement is only marginal, and the evaluation does not cover other standard choices of test-time adaptation benchmark.\nThe small effect and narrow evaluation dampen the significance of the proposed techniques even if they are novel variations on pseudo-labeling.\nAs there is already work experimenting with exactly how to define pseudo-labels—SHOT, RPL, TAST, and conjugate pseudo-labels—more is needed to advance beyond what has already been done and inform future methods for test-time adaptation.\nI encourage the authors to more fully evaluate the proposed method and explore if its improvements compound with other techniques to achieve more significant accuracy improvements.\n\n**Update after response**: The response and revision provided experiments for domain generalization to cover more datasets and more baselines. Furthermore, an ablation justifies the contribution of meta-learning and not merely sampling the pseudo-labels, and the computational cost has been measured. These results reinforce the empirical significance of the work. However, the significance for test-time adaptation is still an issue, because this work only addresses multi-domain/multi-source training, and so does not evaluation on standard benchmarks for adaptation like the ImageNet variants (-C, -R, -V2, ...) or VisDA-C. As such the recommendation is raised to 5 but not higher.\n\n*Points for Rebuttal*\n\n1. Please clarify the implementation of the test-time adaptation step. How many updates are made per batch? What is the batch size?\n2. Please measure the training computation relative to other methods. This could be measured by some subset of time, FLOPs per step, or number of steps, for example.\n3. Please comment on the lack of results for image corruptions (ImageNet-C, CIFAR-10/100-C) which are provided by most prior papers on test-time adaptation. These experiments would provide an informative and more thorough comparison of the proposed method with the state-of-the-art.\n4. Please relate the novelty and claims of this work to those of confidence regularized self-training (see link under Weaknesses).\n5. Please relate the proposed method and degree of improvement to the conjugate pseudo-labels method (Goyal et al. 2022). This is not a prior published method, so it does not count against this submission, but comparing to it would make this submission more comprehensive.\n\n*Miscellaneous Feedback*\n\n- 1. Introduction\n - \"these two settings\" would be clearer to a broader audience if it identifies domain adaptation (which requires targets during training) and generalization (which does not harness targets during testing).\n - The \"pseudo-labels are more accurate\" are they? are the pseudo-labels more correct in themselves, or better calibrated, or what? they can be more effective, if the adaptation is more accurate, but that does not mean the pseudo-labels are in themselves more accurate\n- 4.3 Comparison\n  - What about consistency regularization/DIRT-T and VAT? Such methods sample virtual \"neighbors\" of each point by penalizing changes to the output on small changes to the input. MEMO likewise updates on augmentations of the input. Do the proposed pseudo-labels compound with these methods and give more improvement?\n", "strength_and_weaknesses": "*Strengths*\n\n- The topic of pseudo-label generation is relevant and applicable to the improvement of test-time adaptation, as a set of existing methods adapt by entropy minimization and/or pseudo-label optimization.\n- The meta-learning approach makes sense, and while there is generality in not requiring specialized training (as other methods like BN and Tent do not), if better accuracy can be achieved with different training then it is worthwhile to pursue it.\n- The exposition of the main idea and its technical details is clear (but see weaknesses for missing implementation detail).\n- The related work provides and apt and concise summarization of published test-time adaptation methods.\n- The evaluation experiments with the standard ResNet-50 and the smaller ResNet-18 to show that results generalize beyond a single architecture (although these two are related).\n- It is nice to see that the proposed variational pseudo-labels improve the accuracy at most steps of adaptation (Figure 2, right) and not only at the end of optimization (Figure 2, left).\n\n*Weaknesses*\n\n- Related work is missing, specifically confidence regularized self-training https://arxiv.org/abs/1908.09822 that likewise addresses the uncertainty and error in pseudo-labeling.\n- The method is not fully described in its implementation: what are the batch sizes, number of updates, and the choice of parameters?\n  (The parameters likely follow those of Tent and other methods, which update the affine parameters in normalization layers, but this should be specified.)\n- The amount of improvement is limited. On the main evaluation dataset of PACS, the improvement is about +2 points for online or offline adaptation against Tent (a common baseline).\n- The evaluation is narrower than most work on test-time adaptation. In particular, the works compared against include more benchmarks, such as ImageNet-C for image corruption (Tent, TAST, SHOT), and VLCS + OfficeHome + Terra Incognita for domain generalization (T3A). Relative to these established options, the chosen Fashion-MNIST and MNIST are toy datasets.\n- The process for model and hyperparameter selection is not explained. How were the learning rates for optimization chosen, for example? Are all of these tuned to the accuracy of adaptation on the test set?\n- The training could be more computationally intensive per step, and may require more steps overall, due to the multiple phases of meta-learning. This is not discussed or quantified.\n  The meta-learning approach is likely more expensive than standard supervised training, but the question is how much? This should be measured empirically.\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "QoPtaWvjA-", "reviewer": "Reviewer_vang", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper discusses a probabilistic view of the adaptation process. Based on that, it introduces a meta-adaptation setting with a variational pseudo-labeling distribution.", "summary_of_the_review": "**Concerns about the formulation**\n- I have some concerns about the probabilistic formulation of the paper. For example, the authors suggest that the distributions $p(\\theta_t|x_t,\\theta_s)$ or $p(\\theta_t|\\hat{y}_tx_t,\\theta_s)$ are posterior distributions. Can the authors clarify and elaborate? What is the prior distribution (of $\\theta_t$)? We can't really have a posterior distribution without talking about the prior.\n- I suggest the authors discuss in detail the generating process (all the priors and likelihood). Meaning, to discuss a joint distribution over $\\theta_t,\\theta_s,x_t,x_s,y_s,y_t$ as well as any other latent variables such as $w_t$, and all the terms that this joint distribution factorized into. Then, condition on the observed variables, we can discuss the posterior distributions and predictive distribution. This would make the paper much clearer and help us understand the formulation more deeply.\n\n**Concerns about the evaluation**\nThe evaluation in the paper seems incomprehensive. \n- First of all, it is beneficial to validate the method on standard online domain adaptation datasets, such as Cifar10/Cifar100/ImageNet Corruption and the VisDA17 dataset. This will help us to have an idea of how the method compares against the literature.\n- I also suggest the authors include stronger baselines. For example, EATA [1], TTT[2] or TTT++[3]\n\n\n[1]  Niu, S., Wu, J., Zhang, Y., Chen, Y., Zheng, S., Zhao, P. &amp; Tan, M.. (2022). Efficient Test-Time Model Adaptation without Forgetting. Proceedings of the 39th International Conference on Machine Learning\n[2] Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. In International conference on machine learning, pages 9229–9248. PMLR, 2020.\n[3] Yuejiang Liu, Parth Kothari, Bastien van Delft, Baptiste Bellot-Gurlet, Taylor Mordan, and Alexandre Alahi.TTT++: When does self-supervised test-time training fail or thrive? Advances in Neural Information Processing Systems, 2021. ", "strength_and_weaknesses": "**Strength**: The paper discusses an interesting view of the online adaptation process (the variational meta-learning setting).\n\n**Weakness**:\n- I have several concerns about the probabilistic view of the adaptation process used in the paper. \n- The empirical evaluation is not weak.\n\nDetails of these weaknesses can be found below.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "DA6iUrbNHK", "reviewer": "Reviewer_6yHt", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper addressed test-time adaptation by incorporating uncertainty. This is inspired by the fact that pseudo labels could be confidently wrong, thus pseudo labels are treated as a distribution and test-time training is carried out on the pseudo labels sampled from the distribution. A meta-learning approach is further proposed to better handle domain shifts between source and target domains.\n", "summary_of_the_review": "Overall, this paper presents some interesting ideas for test-time adaptation. In particular, sampling pseudo labels from categorical distribution is interesting, but awaits more empirical evidence. The meta-learning approach is not clearly explained thus potentially raising the question of whether domain shift must be a prior knowledge. Some commonly adopted experiment settings are missing in this work which limits evaluating the proposed method.", "strength_and_weaknesses": "Strength:\n\n1. Allow pseudo label sampling is a novel idea in TTA.\n\nWeakness:\n\n1. Experiments are not carried out on commonly adopted TTA datasets, e.g. CIFAR10-C, CIFAR100-C, ImageNet-C, VisDA. I would like to know why benchmarking on the above datasets are missing.\n\n2. Meta learning requires a held-out validation set. Does it require the validation set to mimic the potential corruptions or distribution shift in the real target domain? If such knowledge is available does it mean the domain shift is a known priori? I am afraid this violates the assumptions of TTA where the information on target domain should be kept unknown before inference.\n\n3. Since the benefit is potentially contributed by sampling pseudo label from a distribution. It is recommended to evaluate directly sampling pseudo label from the posterior, i.e. the probablistic output of unlabeled data, on the presented datasets.\n\n4. I am wondering how integrating the features of neighboring target samples can be achieved in an online TTA mode. If test samples arrive in a stream neighboring samples can not be collected before they are seen.\n\n5. Is the meta-learning adaptation a necessary step? If not, it is recommended to separately evaluate the importance of incorporating the meta-learning adaptation.\n\n6. One important concern for test-time training is the inference speed. How much extra computation overhead does this variational pseudo label introduce?\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "-guXWqunE3", "reviewer": "Reviewer_ZtPj", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a test-time adaptation (TTA) method from probabilistic perspective. Specifically, they analyze first analyze the pseudo labeling, which is a naive approach in TTA, from probabilistic point of view. Based on the analysis, they propose variational pseudo labels,　and meta-learning based algorithms. The empirical validation is conducted on PACS, rotated MNIST and Fashion-MNIST. \n", "summary_of_the_review": "While I agree that the pseudo-label in TTA is worth investigating and derived meta-learning based algorithms is interesting, the paper in current form lacks comprehensive empirical results and discussion about the relationships with prior works. ", "strength_and_weaknesses": "[Strength]\n(1) Test-time adaptation is timely and relevant topic for the conference. \n(2) The paper is well written and easy to follow. \n(3) Meta-learning based algorithms with pseudo label is novel and interesting. \n\n\n[Weaknesses]\n(1) Empirical validation is limited to small-scale dataset. While I agree PACS and rotated MNIST are standard benchmarks, but there are lot of more benchmark dataset (e.g., DomainNet, OfficeHome, TerraIncognita, etc. [1]). Also, it lacks comparison with standard DG methods. Since the standardized evaluation is regarded as one of the big issue in DG papers [1] and this paper does not provide theoretical results, I strongly recommend adding more empirical evaluations to fully show the benefit of the proposal. \n\n[1] Gulrajani, Ishaan and David Lopez-Paz. “In Search of Lost Domain Generalization.” ICLR2021\n\n(2) There are some missing literatures about pseudo labeling both inside [2, 3] and outside [4] the field of the test-time adaptation. Since the pseudo label is the core of the paper, authors should discuss the differences between these studies. Besides, it is also preferable to add these methods as baselines. \n\n[2] Goyal, Sachin et al. “Test-Time Adaptation via Conjugate Pseudo-labels.” NeurIPS2022\n[3] Wang, Qin, Olga Fink, Luc Van Gool, and Dengxin Dai. 2022. “Continual Test-Time Domain Adaptation.” CVPR2022\n[4] Rizve, Mamshad Nayeem et al. “In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning.” ICLR2021", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject