Original: V ARIATIONAL P SEUDO L ABELS FOR M ETA T EST - TIME A DAPTATION
Revised: V ARIATIONAL P SEUDO L ABELS FOR M ETA T EST - TIME A DAPTATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Test-time model adaptation has shown great effectiveness in generalizing over domain shifts.
Revised: Test-time model adaptation has shown great effectiveness in generalizing over domain shifts.

Original: A most successful tactic for test-time adaptation conducts further optimization on the target data using the predictions by the source-trained model.
Revised: A most successful tactic for test-time adaptation conducts further optimization on the target data using the predictions by the source-trained model.

Original: However, due to domain shifts, the source-trained model predictions themselves can be largely inaccurate, which results in a model misspecified to the target data and therefore damages their adaptation ability.
Revised: However, due to domain shifts, the source-trained model predictions themselves can be largely inaccurate, which results in a model misspecified to the target data and therefore damages their adaptation ability.

Original: In this paper, we address test-time adaptation from a probabilistic perspective.
Revised: In this paper, we address test-time adaptation from a probabilistic perspective.

Original: We formulate model adaption as a probabilistic inference problem, which incorporates the uncertainty into source model predictions by modeling pseudo labels as distributions.
Revised: We formulate model adaption as a probabilistic inference problem, which incorporates the uncertainty into source model predictions by modeling pseudo labels as distributions.

Original: Based on the probabilistic formalism, we propose variational pseudo labels that explore the information of neighboring target samples to improve pseudo labels and achieve a model better specified to target data.
Revised: Based on the probabilistic formalism, we propose variational pseudo labels that explore the information of neighboring target samples to improve pseudo labels and achieve a model better specified to target data.

Original: By a meta-learning paradigm, we train our model by simulating domain shifts and the test-time adaptation procedure.
Revised: By a meta-learning paradigm, we train our model by simulating domain shifts and the test-time adaptation procedure.

Original: In doing so, our model learns the ability to generate more accurate pseudo-label distributions and to adapt to new domains.
Revised: In doing so, our model learns the ability to generate more accurate pseudo-label distributions and to adapt to new domains.

Original: Experiments on five widely used datasets demonstrate the effectiveness of our proposal.
Revised: Experiments on five widely used datasets demonstrate the effectiveness of our proposal.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep neural networks start to exhibit generalizability problems and suffer from performance degradation as soon as test data distributions differ from the ones experienced during training, (Geirhos et al., 2018; Recht et al., 2019).
Revised: Deep neural networks start to exhibit generalizability problems and suffer from performance degradation as soon as test data distributions differ from the ones experienced during training, (Geirhos et al., 2018; Recht et al., 2019).

Original: To deal with the distribution shift, domain adaptation, e.g., (Saenko et al., 2010; Long et al., 2015; Lu et al., 2020; Li et al., 2021) and domain generalization, e.g., (Muandet et al., 2013; Motiian et al., 2017; Li et al., 2017; 2020) have proven effective tactics.
Revised: To deal with the distribution shift, domain adaptation, e.g., (Saenko et al., 2010; Long et al., 2015; Lu et al., 2020; Li et al., 2021) and domain generalization, e.g., (Muandet et al., 2013; Motiian et al., 2017; Li et al., 2017; 2020) have proven effective tactics.

Original: However, these two settings either require a large number of (unlabeled) target data during training or do not consider any target information during generalization at all.
Revised: However, these two settings either require a large number of (unlabeled) target data during training or do not consider any target information during generalization at all.

Original: Both of which are not necessarily valid assumptions in realistic scenarios.
Revised: Both of which are not necessarily valid assumptions in realistic scenarios.

Original: Test-time adaptation, e.g., (Sun et al., 2020; Varsavsky et al., 2020; Wang et al., 2021) goes beyond these two setting and introduces a new learning paradigm, which trains a model on source data and further optimizes it using the unlabeled target data at test time to adapt to the target domain.
Revised: Test-time adaptation, e.g., (Sun et al., 2020; Varsavsky et al., 2020; Wang et al., 2021) goes beyond these two setting and introduces a new learning paradigm, which trains a model on source data and further optimizes it using the unlabeled target data at test time to adapt to the target domain.

Original: One widely applied strategy for test-time adaptation updates model parameters by self-supervision (Liang et al., 2020;
Revised: One widely applied strategy for test-time adaptation updates model parameters by self-supervision (Liang et al., 2020;

Original: Wang et al., 2021; Iwasawa & Matsuo, 2021;
Revised: Wang et al., 2021; Iwasawa & Matsuo, 2021;

Original: Niu et al., 2022).
Revised: Niu et al., 2022).

Original: However, due to domain shifts, the source-model predictions on the target samples can be uncertain and inaccurate.
Revised: However, due to domain shifts, the source-model predictions on the target samples can be uncertain and inaccurate.

Original: As self-supervision-based test-time adaptation is often achieved by optimization with pseudo labels or entropy minimization based on the source-trained model predictions, the model can be overconfident on some mispredictions.
Revised: As self-supervision-based test-time adaptation is often achieved by optimization with pseudo labels or entropy minimization based on the source-trained model predictions, the model can be overconfident on some mispredictions.

Original: As a result, the adapted model becomes unreliable and misspecified (Wilson & Izmailov, 2020) to the target data.
Revised: As a result, the adapted model becomes unreliable and misspecified (Wilson & Izmailov, 2020) to the target data.

Original: In this paper we make three contributions.
Revised: In this paper we make three contributions.

Original: First, we address test-time adaptation in a probabilistic framework by formulating it as a variational inference problem.
Revised: First, we address test-time adaptation in a probabilistic framework by formulating it as a variational inference problem.

Original: We define pseudo labels as stochastic variables and estimate a distribution over them by variational inference.
Revised: We define pseudo labels as stochastic variables and estimate a distribution over them by variational inference.

Original: By doing so, the uncertainty in source-trained model predictions is incorporated into the adaptation to the target data at test time.
Revised: By doing so, the uncertainty in source-trained model predictions is incorporated into the adaptation to the target data at test time.

Original: Second, thanks to the proposed probabilistic formalism, it is natural and convenient to utilize variational distributions to leverage extra information.
Revised: Second, thanks to the proposed probabilistic formalism, it is natural and convenient to utilize variational distributions to leverage extra information.

Original: By hinging on this benefit, we design the variational pseudo labels to explore the neighboring information of target samples into the inference of the pseudo label distributions.
Revised: By hinging on this benefit, we design the variational pseudo labels to explore the neighboring information of target samples into the inference of the pseudo label distributions.

Original: By doing so, the variational pseudo labels are more accurate, which enables the source-trained model to be better specified to target data and therefore conducive to model adaptation.
Revised: By doing so, the variational pseudo labels are more accurate, which enables the source-trained model to be better specified to target data and therefore conducive to model adaptation.

Original: Third, we adopt a meta-learning paradigm for optimization to simulate test-time adaptation on source domains.
Revised: Third, we adopt a meta-learning paradigm for optimization to simulate test-time adaptation on source domains.

Original: More specifically, the model is exposed to domain shifts iteratively and optimized to learn the ability of adapting to unseen domains.
Revised: More specifically, the model is exposed to domain shifts iteratively and optimized to learn the ability of adapting to unseen domains.

Original: We conduct experiments on three widely-used benchmarks to demonstrate the promise and effectiveness of our method for test-time adaptation.
Revised: We conduct experiments on three widely-used benchmarks to demonstrate the promise and effectiveness of our method for test-time adaptation.

Original:  M ETHODOLOGY 2. P RELIMINARY
Revised:  M ETHODOLOGY 2. P RELIMINARY

Original: We were given data from different domains defined on the joint space X ×Y , where X and Y denote the data space and label space, respectively.
Revised: We were given data from different domains defined on the joint space X ×Y , where X and Y denote the data space and label space, respectively.

Original: The domains are split into several source domains
Revised: The domains are split into several source domains

Original: D s = (cid:8) ( x s , y s ) i (cid:9) N s i =1 and target domains D t
Revised: D s = (cid:8) ( x s , y s ) i (cid:9) N s i =1 and target domains D t

Original: = (cid:8) ( x t , y t )
Revised: = (cid:8) ( x t , y t )

Original: i (cid:9) N t i =1 .
Revised: i (cid:9) N t i =1 .

Original: The goal is to train a model on source domains that is expected to generalize well on the (unseen) target domains.
Revised: The goal is to train a model on source domains that is expected to generalize well on the (unseen) target domains.

Original: To this end, test-time adaptation methods, e.g., (Wang et al., 2021; Zhang et al., 2021;
Revised: To this end, test-time adaptation methods, e.g., (Wang et al., 2021; Zhang et al., 2021;

Original: Niu et al., 2022), have recently been proposed.
Revised: Niu et al., 2022), have recently been proposed.

Original: These methods adapt the source-trained model by optimization to target domains at test time.
Revised: These methods adapt the source-trained model by optimization to target domains at test time.

Original: A common strategy in these methods is that the model θ is first trained on source data D s by minimizing a supervised loss L train ( θ )=
Revised: A common strategy in these methods is that the model θ is first trained on source data D s by minimizing a supervised loss L train ( θ )=

Original: E ( x s , y s )
Revised: E ( x s , y s )

Original: i ∈D s
Revised: i ∈D s

Original: [ L CE ( x s , y s ; θ )] ; and then at test time they adapt the model θ s to the target domain by optimization with certain surrogate losses, e.g., entropy minimization, based on unlabeled test data, which is formulated as:
Revised: [ L CE ( x s , y s ; θ )] ; and then at test time they adapt the model θ s to the target domain by optimization with certain surrogate losses, e.g., entropy minimization, based on unlabeled test data, which is formulated as:

Original: [Equation]
Revised: [Equation]

Original: where the entropy is calculated on the source model predictions.
Revised: where the entropy is calculated on the source model predictions.

Original: However, test samples from the target domain could be largely misclassified by the source model due to the domain shift, resulting in large uncertainty in the predictions.
Revised: However, test samples from the target domain could be largely misclassified by the source model due to the domain shift, resulting in large uncertainty in the predictions.

Original: Moreover, the entropy minimization tends to update the model with high confidence even for the wrong predictions, which would cause a misspecified model for the target domain.
Revised: Moreover, the entropy minimization tends to update the model with high confidence even for the wrong predictions, which would cause a misspecified model for the target domain.

Original: To solve those problems, in this work we address test-time model adaptation from a probabilistic perspective.
Revised: To solve those problems, in this work we address test-time model adaptation from a probabilistic perspective.

Original: We propose a probabilistic inference framework that models the uncertainty of the source-model predictions by defining distributions over pseudo labels.
Revised: We propose a probabilistic inference framework that models the uncertainty of the source-model predictions by defining distributions over pseudo labels.

Original: Moreover, under the probabilistic formalism, we propose designing variational pseudo labels, which enables the model to incorporate the neighboring information in test samples to combat domain shifts.
Revised: Moreover, under the probabilistic formalism, we propose designing variational pseudo labels, which enables the model to incorporate the neighboring information in test samples to combat domain shifts.

Original: We adopt a metalearning paradigm for optimization, which simulates the domain shifts and adaptation procedure.
Revised: We adopt a metalearning paradigm for optimization, which simulates the domain shifts and adaptation procedure.

Original: By doing so, the model learns to acquire the ability to further adapt itself with pseudo labels to unseen target domains.
Revised: By doing so, the model learns to acquire the ability to further adapt itself with pseudo labels to unseen target domains.

Original: We provide a graphical illustration to highlight the differences between common test-time adaptation and our proposals in Figure 1.
Revised: We provide a graphical illustration to highlight the differences between common test-time adaptation and our proposals in Figure 1.

Original: 2. P ROBABILISTIC TEST - TIME ADAPTATION WITH LATENT PSEUDO LABELS
Revised: 2. P ROBABILISTIC TEST - TIME ADAPTATION WITH LATENT PSEUDO LABELS

Original: We first provide a probabilistic formulation for test-time adaptation based on pseudo labels.
Revised: We first provide a probabilistic formulation for test-time adaptation based on pseudo labels.

Original: Given the target sample x t and the source-trained model θ s , we would like to make predictions on the target sample.
Revised: Given the target sample x t and the source-trained model θ s , we would like to make predictions on the target sample.

Original: To this end, we formulate the predictive likelihood as follows:
Revised: To this end, we formulate the predictive likelihood as follows:

Original: [Equation]
Revised: [Equation]

Original: where we use the value θ ∗ t obtained by the maximum a posterior (MAP) to approximate the integration (Finn et al., 2018).
Revised: where we use the value θ ∗ t obtained by the maximum a posterior (MAP) to approximate the integration (Finn et al., 2018).

Original: Intuitively, the MAP approximation is interpreted as inferring the posterior over θ t : p ( θ t | x t , θ s )
Revised: Intuitively, the MAP approximation is interpreted as inferring the posterior over θ t : p ( θ t | x t , θ s )

Original: ≈ δ
Revised: ≈ δ

Original: ( θ t = θ ∗ t ) , which we obtain by adapting θ s using the target data x t .
Revised: ( θ t = θ ∗ t ) , which we obtain by adapting θ s using the target data x t .

Original: To model the uncertainty of predictions for more robust test-time adaptation, we treat pseudo labels as stochastic variables in the probabilistic framework as shown in Figure 1 (b).
Revised: To model the uncertainty of predictions for more robust test-time adaptation, we treat pseudo labels as stochastic variables in the probabilistic framework as shown in Figure 1 (b).

Original: The pseudo labels are obtained from the source model predictions, which follows categorical distributions.
Revised: The pseudo labels are obtained from the source model predictions, which follows categorical distributions.

Original: Then we reformulate eq.
Revised: Then we reformulate eq.

Original: (2) as follows:
Revised: (2) as follows:

Original: [Equation]
Revised: [Equation]

Original: where θ ∗ t is the MAP value of p ( θ t | ˆ y t , x t , θ s ) , which is obtained via gradient descent on the data x t and the corresponding pseudo labels ˆ y t starting from θ s .
Revised: where θ ∗ t is the MAP value of p ( θ t | ˆ y t , x t , θ s ) , which is obtained via gradient descent on the data x t and the corresponding pseudo labels ˆ y t starting from θ s .

Original: The formulation allows us to sample different pseudo labels from the categorical distribution p (ˆ y t ) to adapt the model θ ∗ t , which takes into account the uncertainty of predictions by the source-trained model.
Revised: The formulation allows us to sample different pseudo labels from the categorical distribution p (ˆ y t ) to adapt the model θ ∗ t , which takes into account the uncertainty of predictions by the source-trained model.

Original: By approximating the expectation of p (ˆ y t ) with the argmax function on p (ˆ y t ) , θ ∗ t is obtained by gradient descent based on only a point estimation of the pseudo label p (ˆ y t ) .
Revised: By approximating the expectation of p (ˆ y t ) with the argmax function on p (ˆ y t ) , θ ∗ t is obtained by gradient descent based on only a point estimation of the pseudo label p (ˆ y t ) .

Original: However, due to domain shifts, the argmax value of p (ˆ y t ) is not guaranteed to be always correct.
Revised: However, due to domain shifts, the argmax value of p (ˆ y t ) is not guaranteed to be always correct.

Original: The adaptation then is similar to entropy minimization (eq. 1), where the adapted model can achieve high confidence but wrong predictions of some target samples due to domain shifts.
Revised: The adaptation then is similar to entropy minimization (eq. 1), where the adapted model can achieve high confidence but wrong predictions of some target samples due to domain shifts.

Original: For example, consider a toy binary classification task, where the predicted probability is [0 . 4 , 0 . 6] with the ground-truth label [1 , 0] .
Revised: For example, consider a toy binary classification task, where the predicted probability is [0 . 4 , 0 . 6] with the ground-truth label [1 , 0] .

Original: The pseudo label generated by selecting the maximum probability is [0 , 1] , which is inaccurate.
Revised: The pseudo label generated by selecting the maximum probability is [0 , 1] , which is inaccurate.

Original: Optimization based on these labels would give rise to a model misspecified to target data, failing to adapt to the target domain.
Revised: Optimization based on these labels would give rise to a model misspecified to target data, failing to adapt to the target domain.

Original: In contrast, our probabilistic formulation allows us to sample pseudo labels from the categorical distribution p (ˆ y t | x t , θ s ) , which incorporates the uncertainty of the pseudo label ˆ y t in a principled way.
Revised: In contrast, our probabilistic formulation allows us to sample pseudo labels from the categorical distribution p (ˆ y t | x t , θ s ) , which incorporates the uncertainty of the pseudo label ˆ y t in a principled way.

Original: Again using the above example, the pseudo label sampled from the predicted distribution has a probability of 40% to be the ground-truth label, which leads to the adaptation of the model in the correct direction.
Revised: Again using the above example, the pseudo label sampled from the predicted distribution has a probability of 40% to be the ground-truth label, which leads to the adaptation of the model in the correct direction.

Original: Therefore, our formulation achieves better adaptation by accessing accurate pseudo labels from the inaccurate prediction distributions.
Revised: Therefore, our formulation achieves better adaptation by accessing accurate pseudo labels from the inaccurate prediction distributions.

Original: 2. V ARIATIONAL PSEUDO LABELS
Revised: 2. V ARIATIONAL PSEUDO LABELS

Original: Under the probabilistic formalism, we derive variational inference of pseudo labels.
Revised: Under the probabilistic formalism, we derive variational inference of pseudo labels.

Original: To train the ability of the model to generate better variational pseudo labels and to fully utilize the pseudo label distributions for better adaptation, we adopt the meta-learning paradigm to simulate domain shifts and test-time adaptation procedures (Finn et al., 2017; Dou et al., 2019; Xiao et al., 2022).
Revised: To train the ability of the model to generate better variational pseudo labels and to fully utilize the pseudo label distributions for better adaptation, we adopt the meta-learning paradigm to simulate domain shifts and test-time adaptation procedures (Finn et al., 2017; Dou et al., 2019; Xiao et al., 2022).

Original: We split the source domains D s into meta-source domains D s ′ and a meta-target domain D t ′ during training.
Revised: We split the source domains D s into meta-source domains D s ′ and a meta-target domain D t ′ during training.

Original: The meta-target domain is selected randomly in each iteration to mimic diverse domain shifts.
Revised: The meta-target domain is selected randomly in each iteration to mimic diverse domain shifts.

Original: To simulate the test-time adaptation and estimation procedure, we maximize the log-likelihood of the meta-target samples after model adaptation on the meta-target data:
Revised: To simulate the test-time adaptation and estimation procedure, we maximize the log-likelihood of the meta-target samples after model adaptation on the meta-target data:

Original: [Equation]
Revised: [Equation]

Original: where p (ˆ y t ′ | x t ′ , θ s ′ ) denotes the distribution of pseudo labels generated by the meta-source model θ s ′ on the meta-target data x t ′ .
Revised: where p (ˆ y t ′ | x t ′ , θ s ′ ) denotes the distribution of pseudo labels generated by the meta-source model θ s ′ on the meta-target data x t ′ .

Original: θ ∗ t ′ is the MAP value of p ( θ t ′ | ˆ y t ′ , x t ′ , θ s ′ ) similar to eq.
Revised: θ ∗ t ′ is the MAP value of p ( θ t ′ | ˆ y t ′ , x t ′ , θ s ′ ) similar to eq.

Original: (3), which is learned to mimic the test-time adaptation procedure.
Revised: (3), which is learned to mimic the test-time adaptation procedure.

Original: Under the meta-learning setting, the actual labels y t ′ of the meta-target data is accessible since source data are fully labeled as shown in Figure 1 (c).
Revised: Under the meta-learning setting, the actual labels y t ′ of the meta-target data is accessible since source data are fully labeled as shown in Figure 1 (c).

Original: We then simulate the test evaluation procedure and further supervise the adapted model θ ∗ t ′ on its meta-target predictions by the actual labels.
Revised: We then simulate the test evaluation procedure and further supervise the adapted model θ ∗ t ′ on its meta-target predictions by the actual labels.

Original: The maximization of the log-likelihood of p ( y t ′ | x t ′ , θ ∗ t ′ ) is realised by a cross-entropy loss on the meta-target predictions and the actual meta-target labels.
Revised: The maximization of the log-likelihood of p ( y t ′ | x t ′ , θ ∗ t ′ ) is realised by a cross-entropy loss on the meta-target predictions and the actual meta-target labels.

Original: Intuitively, the pseudo-label adapted model is supervised to achieve good performance on the adapted data.
Revised: Intuitively, the pseudo-label adapted model is supervised to achieve good performance on the adapted data.

Original: Thus, it learns the ability to generate better pseudo labels and achieve better adaptation across domain shifts with these pseudo labels on new unseen domains.
Revised: Thus, it learns the ability to generate better pseudo labels and achieve better adaptation across domain shifts with these pseudo labels on new unseen domains.

Original: Variational pseudo labels.
Revised: Variational pseudo labels.

Original: Moreover, we also propose variational pseudo labels that incorporate information of the neighboring target samples to estimate pseudo label distributions that are more robust against domain shifts.
Revised: Moreover, we also propose variational pseudo labels that incorporate information of the neighboring target samples to estimate pseudo label distributions that are more robust against domain shifts.

Original: The variational pseudo labels is natural and convenient to deployed under the probabilistic formulation.
Revised: The variational pseudo labels is natural and convenient to deployed under the probabilistic formulation.

Original: Assume that we have a batch of meta-target data X t ′ =
Revised: Assume that we have a batch of meta-target data X t ′ =

Original: (cid:8) x it ′ (cid:9)
Revised: (cid:8) x it ′ (cid:9)

Original: M i
Revised: M i

Original: =1 , we reformulate eq.
Revised: =1 , we reformulate eq.

Original: (4) as:
Revised: (4) as:

Original: [Equation]
Revised: [Equation]

Original: where θ ∗ t ′ is the MAP value of p ( θ t ′ | ˆ y t ′ , x t ′ , θ s ′ ) .
Revised: where θ ∗ t ′ is the MAP value of p ( θ t ′ | ˆ y t ′ , x t ′ , θ s ′ ) .

Original: We introduce the latent variable w t ′ to integrate the information of the neighboring target samples X t ′ as shown in Figure 1.
Revised: We introduce the latent variable w t ′ to integrate the information of the neighboring target samples X t ′ as shown in Figure 1.

Original: To approximate the true posterior of the joint distribution p (ˆ y t ′ , w t ′ ) , we introduce a variational posterior q (ˆ y t ′ , w t ′ | x t ′ , θ s ′ , X t ′ , Y t ′ ) , where Y t ′ =
Revised: To approximate the true posterior of the joint distribution p (ˆ y t ′ , w t ′ ) , we introduce a variational posterior q (ˆ y t ′ , w t ′ | x t ′ , θ s ′ , X t ′ , Y t ′ ) , where Y t ′ =

Original: (cid:8)
Revised: (cid:8)

Original: y it ′ (cid:9) M i
Revised: y it ′ (cid:9) M i

Original: =1 denotes the actual labels of the metatarget data X t ′ .
Revised: =1 denotes the actual labels of the metatarget data X t ′ .

Original: To facilitate the estimation of pseudo labels, we set the prior distribution as:
Revised: To facilitate the estimation of pseudo labels, we set the prior distribution as:

Original: [Equation]
Revised: [Equation]

Original: where p ϕ ( w t ′ | θ s ′ , X t ′ ) is generated by the features of X t ′ together with their output values based on θ s ′ .
Revised: where p ϕ ( w t ′ | θ s ′ , X t ′ ) is generated by the features of X t ′ together with their output values based on θ s ′ .

Original: Similarly, we define the variational posterior distribution as:
Revised: Similarly, we define the variational posterior distribution as:

Original: [Equation]
Revised: [Equation]

Original: where q ϕ ( w t ′ | θ s ′ , X t ′ , Y t ′ ) is obtained by the features of X t ′ and the actual labels Y t ′ based on θ s ′ .
Revised: where q ϕ ( w t ′ | θ s ′ , X t ′ , Y t ′ ) is obtained by the features of X t ′ and the actual labels Y t ′ based on θ s ′ .

Original: By introducing eqs.
Revised: By introducing eqs.

Original: (6) and (7) into (5), we derive the evidence lower bound (ELBO) of the loglikelihood in eq.
Revised: (6) and (7) into (5), we derive the evidence lower bound (ELBO) of the loglikelihood in eq.

Original: (5) as follows:
Revised: (5) as follows:

Original: [Equation]
Revised: [Equation]

Original: Rather than directly using the meta-source model θ s ′ , we estimate the pseudo labels y t ′ from the latent variable w t ′ , which integrates the features of neighboring target samples.
Revised: Rather than directly using the meta-source model θ s ′ , we estimate the pseudo labels y t ′ from the latent variable w t ′ , which integrates the features of neighboring target samples.

Original: By considering the actual labels Y t ′ , the variational distribution utilizes both the target information and categorical information of the neighboring samples.
Revised: By considering the actual labels Y t ′ , the variational distribution utilizes both the target information and categorical information of the neighboring samples.

Original: Thus, the variational posterior models the distribution of different categories in the target domain more reliably and produces more accurate pseudo labels to improve model adaptation.
Revised: Thus, the variational posterior models the distribution of different categories in the target domain more reliably and produces more accurate pseudo labels to improve model adaptation.

Original: 2. M ETA T EST -T IME A DAPTATION : T RAINING AND INFERENCE
Revised: 2. M ETA T EST -T IME A DAPTATION : T RAINING AND INFERENCE

Original: To mimic domain shifts during training, we split each iteration into meta-source, meta-adaptation, and meta-target to simulate the training stage on source domains, test-time adaptation, and test stage on target data, respectively.
Revised: To mimic domain shifts during training, we split each iteration into meta-source, meta-adaptation, and meta-target to simulate the training stage on source domains, test-time adaptation, and test stage on target data, respectively.

Original: Under the meta-learning paradigm, the model is iteratively exposed to domain shifts and learns the capability to adapt the meta-source model θ s ′ to meta-target MAP θ ∗ t ′ with the variational pseudo labels.
Revised: Under the meta-learning paradigm, the model is iteratively exposed to domain shifts and learns the capability to adapt the meta-source model θ s ′ to meta-target MAP θ ∗ t ′ with the variational pseudo labels.

Original: The parameters in the variational inference model ϕ are jointly optimized in order to generate better pseudo labels against domain shifts.
Revised: The parameters in the variational inference model ϕ are jointly optimized in order to generate better pseudo labels against domain shifts.

Original: Meta-source.
Revised: Meta-source.

Original: We first train and optimize the model on meta-source domains by minimizing the supervised loss:
Revised: We first train and optimize the model on meta-source domains by minimizing the supervised loss:

Original: [Equation]
Revised: [Equation]

Original: where ( x s ′ , y s ′ ) denotes the input-label pairs of samples on meta-source domains.
Revised: where ( x s ′ , y s ′ ) denotes the input-label pairs of samples on meta-source domains.

Original: θ s ′ are the model parameters trained on the meta-source data.
Revised: θ s ′ are the model parameters trained on the meta-source data.

Original: Meta-adaptation.
Revised: Meta-adaptation.

Original: Once the meta-source-trained model θ s ′ is obtained, we generate the pseudo labels p ( ˆ y t ′ | w t ′ , x t ′ ) of the meta-target data with the variational posterior q ϕ ( w t ′ | θ s ′ , X t ′ , Y t ′ ) .
Revised: Once the meta-source-trained model θ s ′ is obtained, we generate the pseudo labels p ( ˆ y t ′ | w t ′ , x t ′ ) of the meta-target data with the variational posterior q ϕ ( w t ′ | θ s ′ , X t ′ , Y t ′ ) .

Original: The test-time adaptation procedure is simulated by obtaining θ ∗ t ′ :
Revised: The test-time adaptation procedure is simulated by obtaining θ ∗ t ′ :

Original: [Equation]
Revised: [Equation]

Original: where λ 1 denotes the learning rate of the optimization in the meta-adaptation stage.
Revised: where λ 1 denotes the learning rate of the optimization in the meta-adaptation stage.

Original: Meta-target.
Revised: Meta-target.

Original: Since our final goal is to obtain good performance on the target data after optimization with pseudo labels, we further mimic the test-time inference on the meta-target domain and supervise the meta-target prediction on θ ∗ t ′ by maximizing the log-likelihood, which is equal to minimizing:
Revised: Since our final goal is to obtain good performance on the target data after optimization with pseudo labels, we further mimic the test-time inference on the meta-target domain and supervise the meta-target prediction on θ ∗ t ′ by maximizing the log-likelihood, which is equal to minimizing:

Original: [Equation]
Revised: [Equation]

Original: where y t ′ denotes the ground truth label of x t ′ .
Revised: where y t ′ denotes the ground truth label of x t ′ .

Original: The parameters θ are finally updated by
Revised: The parameters θ are finally updated by

Original: [Equation]
Revised: [Equation]

Original: where λ 2 denotes the learning rate for the meta-target stage.
Revised: where λ 2 denotes the learning rate for the meta-target stage.

Original: Note that the loss in eq.
Revised: Note that the loss in eq.

Original: (11) is computed on the parameters θ ∗ t ′ obtained by eq.
Revised: (11) is computed on the parameters θ ∗ t ′ obtained by eq.

Original: (10), while the optimization is performed over the meta-sourcetrained parameters θ s ′ in eq.
Revised: (10), while the optimization is performed over the meta-sourcetrained parameters θ s ′ in eq.

Original: Intuitively, the parameters are optimized to learn the ability to handle domain shifts, such that adaptation with variational pseudo labels of data from new domains improves the predictions on the new domain.
Revised: Intuitively, the parameters are optimized to learn the ability to handle domain shifts, such that adaptation with variational pseudo labels of data from new domains improves the predictions on the new domain.

Original: The parameters in the variational inference model ϕ are jointly trained with θ .
Revised: The parameters in the variational inference model ϕ are jointly trained with θ .

Original: To guarantee that the variational pseudo labels do extract the neighboring information for discrimination, we add a cross-entropy loss L ˆ ce on the variational pseudo labels and the corresponding actual labels during training.
Revised: To guarantee that the variational pseudo labels do extract the neighboring information for discrimination, we add a cross-entropy loss L ˆ ce on the variational pseudo labels and the corresponding actual labels during training.

Original: Thus, ϕ is updated by:
Revised: Thus, ϕ is updated by:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where λ 3 denotes the learning rates.
Revised: where λ 3 denotes the learning rates.

Original: Test-time adaptation and prediction.
Revised: Test-time adaptation and prediction.

Original: At test time, the model trained on the source domains with the above meta-learning strategy θ s is adapted by further optimization using eq.
Revised: At test time, the model trained on the source domains with the above meta-learning strategy θ s is adapted by further optimization using eq.

Original: The adapted model is then evaluated on the (unseen) target data D t .
Revised: The adapted model is then evaluated on the (unseen) target data D t .

Original: The prediction is formulated as:
Revised: The prediction is formulated as:

Original: [Equation]
Revised: [Equation]

Original: (14) where θ ∗ t is the MAP value of p ( θ t | x t , ˆ y t , θ s ) .
Revised: (14) where θ ∗ t is the MAP value of p ( θ t | x t , ˆ y t , θ s ) .

Original: p ϕ ( w t )=
Revised: p ϕ ( w t )=

Original: p ϕ ( w t | θ s , X t ) is generated by the features of X t according to the outputs or common pseudo labels based on θ s .
Revised: p ϕ ( w t | θ s , X t ) is generated by the features of X t according to the outputs or common pseudo labels based on θ s .

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Test-time adaptation.
Revised: Test-time adaptation.

Original: By combining the advantages of both domain adaptation (Ganin & Lempitsky, 2015; Long et al., 2015; Hoffman et al., 2018; Lu et al., 2020; Tzeng et al., 2017; Shen et al.,
Revised: By combining the advantages of both domain adaptation (Ganin & Lempitsky, 2015; Long et al., 2015; Hoffman et al., 2018; Lu et al., 2020; Tzeng et al., 2017; Shen et al.,

Original: and domain generalization (Muandet et al., 2013; Li et al., 2017; 2019; Du et al., 2020;
Revised: and domain generalization (Muandet et al., 2013; Li et al., 2017; 2019; Du et al., 2020;

Original: Zhou et al., 2020; 2022), test-time adaptation (Sun et al., 2020; Dubey et al., 2021; Wang et al., 2021;
Revised: Zhou et al., 2020; 2022), test-time adaptation (Sun et al., 2020; Dubey et al., 2021; Wang et al., 2021;

Original: Zhou & Levine, 2021; Chen et al., 2022) and source-free adaptation (Liang et al., 2020; Eastwood et al., 2021) are proposed to train a model only on source domains while adapting it to unlabeled target data at test time.
Revised: Zhou & Levine, 2021; Chen et al., 2022) and source-free adaptation (Liang et al., 2020; Eastwood et al., 2021) are proposed to train a model only on source domains while adapting it to unlabeled target data at test time.

Original: Several methods update normalization statistics of the model to handle domain shifts (Schneider et al., 2020; Du et al., 2021;
Revised: Several methods update normalization statistics of the model to handle domain shifts (Schneider et al., 2020; Du et al., 2021;

Original: Hu et al., 2021).
Revised: Hu et al., 2021).

Original: Sun et al.
Revised: Sun et al.

Original: (2020) proposed to fine-tune the model parameters by a self-supervised loss at test time.
Revised: (2020) proposed to fine-tune the model parameters by a self-supervised loss at test time.

Original: Liu et al.
Revised: Liu et al.

Original: (2021) further enhanced the method by introducing a test-time feature alignment strategy.
Revised: (2021) further enhanced the method by introducing a test-time feature alignment strategy.

Original: Instead of using an extra self-supervised loss, Wang et al.
Revised: Instead of using an extra self-supervised loss, Wang et al.

Original: proposed fully test-time adaptation by entropy minimization, which is followed in several recent works (Zhang et al., 2021; Niu et al., 2022; Jang & Chung, 2022).
Revised: proposed fully test-time adaptation by entropy minimization, which is followed in several recent works (Zhang et al., 2021; Niu et al., 2022; Jang & Chung, 2022).

Original: Zhang et al.
Revised: Zhang et al.

Original: (2021) minimized the entropy of the marginal output distribution averaged over multiple augmentations of a single target sample.
Revised: (2021) minimized the entropy of the marginal output distribution averaged over multiple augmentations of a single target sample.

Original: Niu et al.
Revised: Niu et al.

Original: (2022) proposed an efficient test-time adaptation method without forgetting by adapting with low entropy samples with Fisher regularization.
Revised: (2022) proposed an efficient test-time adaptation method without forgetting by adapting with low entropy samples with Fisher regularization.

Original: Different from these methods, Iwasawa & Matsuo (2021) tried to adjust the classifier with pseudo labels of the target data, without fine-tuning the model parameters.
Revised: Different from these methods, Iwasawa & Matsuo (2021) tried to adjust the classifier with pseudo labels of the target data, without fine-tuning the model parameters.

Original: Different from these methods, we propose a probabilistic formulation of test-time adaptation, which models the uncertainty of pseudo labels for better adaptation.
Revised: Different from these methods, we propose a probabilistic formulation of test-time adaptation, which models the uncertainty of pseudo labels for better adaptation.

Original: We also introduce variational pseudo labels with meta-adaptation to further learn the ability to improve the pseudo labels and adaptations.
Revised: We also introduce variational pseudo labels with meta-adaptation to further learn the ability to improve the pseudo labels and adaptations.

Original: Meta-learning.
Revised: Meta-learning.

Original: Meta-learning-based methods (Alet et al., 2021; Xiao et al., 2022; Goyal et al., 2022) have been studied for test-time adaptation before.
Revised: Meta-learning-based methods (Alet et al., 2021; Xiao et al., 2022; Goyal et al., 2022) have been studied for test-time adaptation before.

Original: Alet et al.
Revised: Alet et al.

Original: (2021) learned to adapt with a contrastive loss.
Revised: (2021) learned to adapt with a contrastive loss.

Original: Goyal et al.
Revised: Goyal et al.

Original: (2022) meta-learned the loss functions of the test-time adaptation for better adaptation.
Revised: (2022) meta-learned the loss functions of the test-time adaptation for better adaptation.

Original: Xiao et al.
Revised: Xiao et al.

Original: (2022) proposed a single sample generalization, which adapts the model to each individual target sample through mimicking domain shifts during training.
Revised: (2022) proposed a single sample generalization, which adapts the model to each individual target sample through mimicking domain shifts during training.

Original: Our method also learns the adaptation ability under the meta-learning setting.
Revised: Our method also learns the adaptation ability under the meta-learning setting.

Original: We design meta-adaptation to simulate the test-time adaptation procedure and supervise it based on our probabilistic formulation.
Revised: We design meta-adaptation to simulate the test-time adaptation procedure and supervise it based on our probabilistic formulation.

Original: We further supervise the meta-adapted model in the meta-target stage to learn the variational pseudo label generation and adaptation ability.
Revised: We further supervise the meta-adapted model in the meta-target stage to learn the variational pseudo label generation and adaptation ability.

Original: Pseudo label learning.
Revised: Pseudo label learning.

Original: In pseudo label learning, the motive is to use a model’s best predictions, e.g., high confidence predictions, and use the corresponding samples and their predictions for retraining the model on a given downstream task.
Revised: In pseudo label learning, the motive is to use a model’s best predictions, e.g., high confidence predictions, and use the corresponding samples and their predictions for retraining the model on a given downstream task.

Original: Tasks examples include classification (Yalniz et al., 2019;
Revised: Tasks examples include classification (Yalniz et al., 2019;

Original: Xie et al., 2020), segmentation (Zou et al., 2020), and object detection (Li et al., 2022).
Revised: Xie et al., 2020), segmentation (Zou et al., 2020), and object detection (Li et al., 2022).

Original: Pham et al.
Revised: Pham et al.

Original: (2021) addresses the problem of confirmation bias in pseudo labeling and utilize a teacher-student network for the task of image classification.
Revised: (2021) addresses the problem of confirmation bias in pseudo labeling and utilize a teacher-student network for the task of image classification.

Original: (Zou et al., 2019) utilize the output logits of softmax as the prediction probability and train the model to directly maximize the logits.
Revised: (Zou et al., 2019) utilize the output logits of softmax as the prediction probability and train the model to directly maximize the logits.

Original: (Wang et al., 2022) utilizes confidence to approximate the domain difference and applies augmentations to improve quality if below the threshold to update the student model.
Revised: (Wang et al., 2022) utilizes confidence to approximate the domain difference and applies augmentations to improve quality if below the threshold to update the student model.

Original: (Rizve et al., 2021) assumes that there is access to target labels in a semi-supervised learning and relies on prediction uncertainties leveraged through the labeled samples in target to and generates new pseudo labels.
Revised: (Rizve et al., 2021) assumes that there is access to target labels in a semi-supervised learning and relies on prediction uncertainties leveraged through the labeled samples in target to and generates new pseudo labels.

Original: For semi-supervised learning, (Miyato et al., 2018) proposes a regularization method that uses target data without labels.
Revised: For semi-supervised learning, (Miyato et al., 2018) proposes a regularization method that uses target data without labels.

Original: Shu et al.
Revised: Shu et al.

Original: (2018) constructs a new source domain using pseudo labelling using available target data during source training.
Revised: (2018) constructs a new source domain using pseudo labelling using available target data during source training.

Original: For domain generalization and offline adaptation, Abdo et al.
Revised: For domain generalization and offline adaptation, Abdo et al.

Original: (2009) utilize pseudo labels from network extracting features and depth features.
Revised: (2009) utilize pseudo labels from network extracting features and depth features.

Original: For online adaptation, Chen et al.
Revised: For online adaptation, Chen et al.

Original: (2022) utilize a contrastive method and utilize pseudo labels to ensure that same class negative samples are not used in contrastive loss optimization.
Revised: (2022) utilize a contrastive method and utilize pseudo labels to ensure that same class negative samples are not used in contrastive loss optimization.

Original: For semi-supervised adaptation Zhou et al.
Revised: For semi-supervised adaptation Zhou et al.

Original: (2021) utilize pseudo labels and use confidence as a criteria.
Revised: (2021) utilize pseudo labels and use confidence as a criteria.

Original: In contrast to existing works, our method uses meta learning and is probabilistic in nature for test-time adaptation.
Revised: In contrast to existing works, our method uses meta learning and is probabilistic in nature for test-time adaptation.

Original:  E XPERIMENTS 4. S ETTINGS
Revised:  E XPERIMENTS 4. S ETTINGS

Original: Five datasets.
Revised: Five datasets.

Original: We demonstrate the effectiveness of our method on image classification and domain generalization settings.
Revised: We demonstrate the effectiveness of our method on image classification and domain generalization settings.

Original: We evaluate our method on five widely used datasets in domain generalization.
Revised: We evaluate our method on five widely used datasets in domain generalization.

Original: PACS (Li et al., 2017) consists of 7 classes and 4 domains: Photo, Art painting, Cartoon, and Sketch with 9991 samples.
Revised: PACS (Li et al., 2017) consists of 7 classes and 4 domains: Photo, Art painting, Cartoon, and Sketch with 9991 samples.

Original: VLCS (Fang et al., 2013) consists of 5 classes and 4 domains: Pascal, LabelMe, Caltech, SUN with 10,729 samples.
Revised: VLCS (Fang et al., 2013) consists of 5 classes and 4 domains: Pascal, LabelMe, Caltech, SUN with 10,729 samples.

Original: TerraIncognita (Beery et al., 2018) consists of 10 classes and 4 domains: Location 100, Location 38, Location 43 and Location 46 with 24,
Revised: TerraIncognita (Beery et al., 2018) consists of 10 classes and 4 domains: Location 100, Location 38, Location 43 and Location 46 with 24,

Original: [Table]
Revised: [Table]

Original: samples.
Revised: samples.

Original: We follow the training and validation split in (Li et al., 2017) and evaluate the model according to the “leave-one-out” protocol (Li et al., 2019; Carlucci et al., 2019).
Revised: We follow the training and validation split in (Li et al., 2017) and evaluate the model according to the “leave-one-out” protocol (Li et al., 2019; Carlucci et al., 2019).

Original: We also evaluate our method on the Rotated MNIST and Fashion-MNIST datasets following Piratla et al.
Revised: We also evaluate our method on the Rotated MNIST and Fashion-MNIST datasets following Piratla et al.

Original: (2020), where the images are rotated by different angles as different domains.
Revised: (2020), where the images are rotated by different angles as different domains.

Original: We use the subsets with rotation angles from 15 ◦ to 75 ◦ in intervals of 15 ◦ as five source domains, and images rotated by 0 ◦ and 90 ◦ as the target domains.
Revised: We use the subsets with rotation angles from 15 ◦ to 75 ◦ in intervals of 15 ◦ as five source domains, and images rotated by 0 ◦ and 90 ◦ as the target domains.

Original: Two adaptation settings.
Revised: Two adaptation settings.

Original: To demonstrate the effectiveness of our method, we evaluate on both offline and online test-time adaptation settings.
Revised: To demonstrate the effectiveness of our method, we evaluate on both offline and online test-time adaptation settings.

Original: Offline test-time adaptation assumes there are already unlabeled target data available for adaptation.
Revised: Offline test-time adaptation assumes there are already unlabeled target data available for adaptation.

Original: To be close to real-world application where it is difficult to access the whole target set, we use different amounts of target data for offline adaptation.
Revised: To be close to real-world application where it is difficult to access the whole target set, we use different amounts of target data for offline adaptation.

Original: We adapt the model on the available target data and evaluate it on the entire target set without continuously fine-tuning on the entire target set.
Revised: We adapt the model on the available target data and evaluate it on the entire target set without continuously fine-tuning on the entire target set.

Original: We also evaluate our method for online test-time adaptation (Iwasawa & Matsuo, 2021).
Revised: We also evaluate our method for online test-time adaptation (Iwasawa & Matsuo, 2021).

Original: In real-world application, as aforementioned, instead of accessing the entire target set, we usually obtain unlabeled target data in an online manner.
Revised: In real-world application, as aforementioned, instead of accessing the entire target set, we usually obtain unlabeled target data in an online manner.

Original: To achieve continuous adaptation and improvement of the model on target data, we increment the target data iteratively and keep adapting and evaluating the model on the online target data.
Revised: To achieve continuous adaptation and improvement of the model on target data, we increment the target data iteratively and keep adapting and evaluating the model on the online target data.

Original: Implementation details We make use of ResNet-18 for all our experiments and ablation studies and report the accuracies on ResNet-50 for comparison as well.
Revised: Implementation details We make use of ResNet-18 for all our experiments and ablation studies and report the accuracies on ResNet-50 for comparison as well.

Original: The backbones are pretrained on
Revised: The backbones are pretrained on

Original: ImageNet same as the previous methods.
Revised: ImageNet same as the previous methods.

Original: During training, we use a varied learning rate throughout the model.
Revised: During training, we use a varied learning rate throughout the model.

Original: We set the learning rate for the pretrained ResNet to 5e-5 and the learning rate of the variational module and classifiers as 1e-4 for all datasets.
Revised: We set the learning rate for the pretrained ResNet to 5e-5 and the learning rate of the variational module and classifiers as 1e-4 for all datasets.

Original: During test-time adaptation, we utilize a learning rate of 1e-4 for all layers.
Revised: During test-time adaptation, we utilize a learning rate of 1e-4 for all layers.

Original: We intend to release the code on acceptance of this paper.
Revised: We intend to release the code on acceptance of this paper.

Original: 4. A BLATION STUDIES
Revised: 4. A BLATION STUDIES

Original: Benefits of our probabilistic test-time adaptation We first investigate the effectiveness of our probabilistic formulation of test-time adaptation and its meta-learned variational pseudo labels.
Revised: Benefits of our probabilistic test-time adaptation We first investigate the effectiveness of our probabilistic formulation of test-time adaptation and its meta-learned variational pseudo labels.

Original: To demonstrate the benefits of the probabilistic formulation, we conduct test-time adaptation with eq.
Revised: To demonstrate the benefits of the probabilistic formulation, we conduct test-time adaptation with eq.

Original: and compare it with a common test-time adaptation tactic (Wang et al., 2021), as in eq.
Revised: and compare it with a common test-time adaptation tactic (Wang et al., 2021), as in eq.

Original: As shown in Table 1, both adaptation with and without probabilistic formulation achieve good improvements over the ERM baseline.
Revised: As shown in Table 1, both adaptation with and without probabilistic formulation achieve good improvements over the ERM baseline.

Original: Our probabilistic test-time adaptation performs better than the common one on most target domains for both the online and offline adaptation settings, which demonstrates the benefits of modeling uncertainty during adaptation at test time.
Revised: Our probabilistic test-time adaptation performs better than the common one on most target domains for both the online and offline adaptation settings, which demonstrates the benefits of modeling uncertainty during adaptation at test time.

Original: Moreover, we incorporate the distribution of pseudo labels into the probabilistic formulation, and further propose the variational pseudo labels with meta-adaptation based on the pseudo label distributions.
Revised: Moreover, we incorporate the distribution of pseudo labels into the probabilistic formulation, and further propose the variational pseudo labels with meta-adaptation based on the pseudo label distributions.

Original: As shown in the fourth and last row in Table 1, our method further improves the performance of both online and offline adaptation, demonstrating the effectiveness of the variational pseudo labels with meta-adaptation.
Revised: As shown in the fourth and last row in Table 1, our method further improves the performance of both online and offline adaptation, demonstrating the effectiveness of the variational pseudo labels with meta-adaptation.

Original: With the probabilistic formulation, it is natural and simple to define the problem as a variational inference problem and solve the problem under the meta-learning
Revised: With the probabilistic formulation, it is natural and simple to define the problem as a variational inference problem and solve the problem under the meta-learning

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Table]
Revised: [Table]

Original: framework.
Revised: framework.

Original: The results further demonstrate the benefits and importance of our probabilistic formulations of test-time adaptation.
Revised: The results further demonstrate the benefits and importance of our probabilistic formulations of test-time adaptation.

Original: Benefits of variational pseudo-labels Based on the probabilistic formulation, we introduce variational pseudo labels to incorporate the target information of the neighboring target data for each target sample.
Revised: Benefits of variational pseudo-labels Based on the probabilistic formulation, we introduce variational pseudo labels to incorporate the target information of the neighboring target data for each target sample.

Original: To demonstrate the effectiveness of our variational pseudo label, we compare it with the normal pseudo labels drawn directly from the prediction distributions of source-trained models.
Revised: To demonstrate the effectiveness of our variational pseudo label, we compare it with the normal pseudo labels drawn directly from the prediction distributions of source-trained models.

Original: We evaluate the methods in offline adaptation settings with different amounts of target data.
Revised: We evaluate the methods in offline adaptation settings with different amounts of target data.

Original: As shown in Figure 2 (left), adaptation with our variational pseudo labels achieves better overall results than the normal pseudo labels consistently.
Revised: As shown in Figure 2 (left), adaptation with our variational pseudo labels achieves better overall results than the normal pseudo labels consistently.

Original: We also provide the adaptation results along with adaptation steps in Figure 2 (right).
Revised: We also provide the adaptation results along with adaptation steps in Figure 2 (right).

Original: Starting from the same baseline accuracy, the variational pseudo labels achieve faster adaptation than the normal pseudo labels.
Revised: Starting from the same baseline accuracy, the variational pseudo labels achieve faster adaptation than the normal pseudo labels.

Original: Adaptation with variational pseudo labels is less prone to saturating in performance, leading to better final accuracy.
Revised: Adaptation with variational pseudo labels is less prone to saturating in performance, leading to better final accuracy.

Original: Benefits of meta learning We also investigate the importance of meta-learning in our method.
Revised: Benefits of meta learning We also investigate the importance of meta-learning in our method.

Original: For this experiment, we do not make meta-learning for source training and test-time adaptation and only use variational labels that have been generated for adaptation.
Revised: For this experiment, we do not make meta-learning for source training and test-time adaptation and only use variational labels that have been generated for adaptation.

Original: We observe that meta-learning indeed helps in our formulation.
Revised: We observe that meta-learning indeed helps in our formulation.

Original: Without meta-learning, it is difficult for the model to learn the ability to handle domain shifts.
Revised: Without meta-learning, it is difficult for the model to learn the ability to handle domain shifts.

Original: Thus, there is a significant decrease in accuracy as shown in Table 2.
Revised: Thus, there is a significant decrease in accuracy as shown in Table 2.

Original: Offline adaptation with less target data In real applications, it is difficult to access the entire target set at once for adaptation.
Revised: Offline adaptation with less target data In real applications, it is difficult to access the entire target set at once for adaptation.

Original: Therefore, we estimate our method under the offline test-time adaptation settings with different amounts of target data.
Revised: Therefore, we estimate our method under the offline test-time adaptation settings with different amounts of target data.

Original: As shown in Figure 3, the accuracy increases obviously with small numbers of target samples, e.g., 10% and 25%.
Revised: As shown in Figure 3, the accuracy increases obviously with small numbers of target samples, e.g., 10% and 25%.

Original: However, the overall accuracy tends to saturate when keeping increasing the number of target data for adaptation.
Revised: However, the overall accuracy tends to saturate when keeping increasing the number of target data for adaptation.

Original: This indicates that our method is able to achieve good adaptation under the offline adaptation setting with even small amounts of target data, showing the applicability of the proposed method in practice.
Revised: This indicates that our method is able to achieve good adaptation under the offline adaptation setting with even small amounts of target data, showing the applicability of the proposed method in practice.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: C OMPARISONS
Revised: C OMPARISONS

Original: To further demonstrate the effectiveness of our method
Revised: To further demonstrate the effectiveness of our method

Original: [Table]
Revised: [Table]

Original: we compare our method with some stateof-the-art test-time adaptation methods and standard domain generalization methods.
Revised: we compare our method with some stateof-the-art test-time adaptation methods and standard domain generalization methods.

Original: Table 3 shows the results on PACS, VLCS and TerraIncognita using ResNet-18.
Revised: Table 3 shows the results on PACS, VLCS and TerraIncognita using ResNet-18.

Original: Compared with the other state-ofthe-art domain generalization methods and test-time adaptation methods with the online adaptation setting, our method performs better.
Revised: Compared with the other state-ofthe-art domain generalization methods and test-time adaptation methods with the online adaptation setting, our method performs better.

Original: On datasets such as PACS and TerraIncognita, we are better by state-of-the-art methods significantly.
Revised: On datasets such as PACS and TerraIncognita, we are better by state-of-the-art methods significantly.

Original: We also report results on the PACS dataset using ResNetas the backbone, the performance of our method is competitive and better than most of the state-ofthe-art methods.
Revised: We also report results on the PACS dataset using ResNetas the backbone, the performance of our method is competitive and better than most of the state-ofthe-art methods.

Original: We provide the detailed comparison in Appendix 5.
Revised: We provide the detailed comparison in Appendix 5.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We propose to cast test-time adaptation as a probabilistic inference problem and model pseudolabels as distributions in the formulation.
Revised: We propose to cast test-time adaptation as a probabilistic inference problem and model pseudolabels as distributions in the formulation.

Original: By modeling the uncertainty into the pseudo label distributions, the probabilistic formulation mitigates adaptation with inaccurate pseudo labels or predictions, which arises due to domain shifts and lead to misspecified models after adaptation.
Revised: By modeling the uncertainty into the pseudo label distributions, the probabilistic formulation mitigates adaptation with inaccurate pseudo labels or predictions, which arises due to domain shifts and lead to misspecified models after adaptation.

Original: Based on the probabilistic formulation, we further propose variational pseudo labels under the meta-adaptation paradigm, which exposes the model to domain shifts and learns the ability to adapt with pseudo labels incorporating target information of the neighboring target samples.
Revised: Based on the probabilistic formulation, we further propose variational pseudo labels under the meta-adaptation paradigm, which exposes the model to domain shifts and learns the ability to adapt with pseudo labels incorporating target information of the neighboring target samples.

Original: Ablation studies and further comparisons show the effectiveness of our method on five common domain generalization datasets.
Revised: Ablation studies and further comparisons show the effectiveness of our method on five common domain generalization datasets.

Original: A D ETAILED FORMULATION
Revised: A D ETAILED FORMULATION

Original: We start the objective function from p ( y t ′ | x t ′ , θ s ′ , X t ′ ) .
Revised: We start the objective function from p ( y t ′ | x t ′ , θ s ′ , X t ′ ) .

Original: Here we provide the detailed generating process of the formulation:
Revised: Here we provide the detailed generating process of the formulation:

Original: [Equation]
Revised: [Equation]

Original: We then introduce the pseudo labels ˆ y t ′ as the latent variable into eq.
Revised: We then introduce the pseudo labels ˆ y t ′ as the latent variable into eq.

Original: (15) and derive it as:
Revised: (15) and derive it as:

Original: [Equation]
Revised: [Equation]

Original: Theoretically, the distribution p ( θ t ′ ) is obtained by p ( θ t ′ | y t ′ , x t ′ , θ s ′ )
Revised: Theoretically, the distribution p ( θ t ′ ) is obtained by p ( θ t ′ | y t ′ , x t ′ , θ s ′ )

Original: ∝ p ( y t | x t , θ t ) p ( θ t | θ s ) , where p ( θ t | θ s ) is the prior distribution.
Revised: ∝ p ( y t | x t , θ t ) p ( θ t | θ s ) , where p ( θ t | θ s ) is the prior distribution.

Original: To simplify the formulation, we approximate the integration of p ( θ t ′ ) by the maximum a posterior (MAP) value of θ ∗ t ′ .
Revised: To simplify the formulation, we approximate the integration of p ( θ t ′ ) by the maximum a posterior (MAP) value of θ ∗ t ′ .

Original: We obtain the MAP value by training the model θ with inputs x t ′ and pseudo labels y t ′ starting from θ s ′ .
Revised: We obtain the MAP value by training the model θ with inputs x t ′ and pseudo labels y t ′ starting from θ s ′ .

Original: The formulation then is derived as:
Revised: The formulation then is derived as:

Original: [Equation]
Revised: [Equation]

Original: To obtain better pseudo labels ˆ y t ′ , we further introduce the latent variable w t ′ into eq.
Revised: To obtain better pseudo labels ˆ y t ′ , we further introduce the latent variable w t ′ into eq.

Original: (17) and a variational posterior of the joint distribution q (ˆ y t ′ , w t ′ ) .
Revised: (17) and a variational posterior of the joint distribution q (ˆ y t ′ , w t ′ ) .

Original: The formulation is then derived as: p ( y t ′ | x t ′ , θ s ′ , X t ′)
Revised: The formulation is then derived as: p [Equation] y t ′ | x t ′ , θ s ′ , X t ′

Original: [Equation]
Revised: [Equation]

Original: where q (ˆ y t ′ , w t ′ | x t ′ , θ s ′ , X t ′ , Y t ′ ) and p (ˆ y t ′ , w t ′
Revised: where q (ˆ y t ′ , w t ′ | x t ′ , θ s ′ , X t ′ , Y t ′ ) and p (ˆ y t ′ , w t ′

Original: | x t ′ , θ s ′ , X t ′ )
Revised: | x t ′ , θ s ′ , X t ′ )

Original: denote the prior and posterior distributions, respectively.
Revised: denote the prior and posterior distributions, respectively.

Original: B I MPLEMENTATION DETAILS
Revised: B I MPLEMENTATION DETAILS

Original: Our train setup follows Iwasawa & Matsuo (2021).
Revised: Our train setup follows Iwasawa & Matsuo (2021).

Original: We use a batch size of 70 and train our method using the ERM algorithm Gulrajani & Lopez-Paz (2020).
Revised: We use a batch size of 70 and train our method using the ERM algorithm Gulrajani & Lopez-Paz (2020).

Original: As stated, our backbones such as ResNet18 and ResNet-34 are pretrained on ImageNet same as the previous methods.
Revised: As stated, our backbones such as ResNet18 and ResNet-34 are pretrained on ImageNet same as the previous methods.

Original: During our training, the model with highest validation accuracy is selected for adaptation on the target domain.
Revised: During our training, the model with highest validation accuracy is selected for adaptation on the target domain.

Original: We use similar settings for all Domain Generalization benchmarks that have been reported in the paper.
Revised: We use similar settings for all Domain Generalization benchmarks that have been reported in the paper.

Original: We train all our models on NVIDIA Tesla 1080Ti GPU.
Revised: We train all our models on NVIDIA Tesla 1080Ti GPU.

Original: In Table 7 we report the runtime required for test-time adaptation.
Revised: In Table 8 we report the runtime required for test-time adaptation.

Original: We observe that our method is comparable to methods that are not optimization free such as SHOT and PL.
Revised: We observe that our method is comparable to methods that are not optimization free such as SHOT and PL.

Original: We also in Table 8 report the amount of time consumed during source training.
Revised: We also in Table 7 report the amount of time consumed during source training.

Original: C D ETAILED EXPERIMENTAL RESULTS
Revised: C D ETAILED EXPERIMENTAL RESULTS

Revised: Detailed experimental results.

Original: In Table 4 we report our detailed performance and comparison to existing methods on PACS with both ResNet-18 and ResNet-50 as the backbone.
Revised: In Table 4 we report our detailed performance and comparison to existing methods on PACS with both ResNet-18 and ResNet-50 as the backbone.

Original: We observe that our method shows an improvement in accuracy compared to other methods especially on ”Art-Painting”, ”Cartoon” and ”Photo” Domains.
Revised: We observe that our method shows an improvement in accuracy compared to other methods especially on ”ArtPainting”, ”Cartoon” and ”Photo” Domains.

Original: The results show the benefits of using Variational Pseudo labels for adaptation.
Revised: The results show the benefits of using Variational Pseudo labels for adaptation.

Original: We also conduct experiments on rotated MNIST and rotated Fashion-MNIST for comparison as shown in Table 5.
Revised: We also conduct experiments on rotated MNIST and rotated Fashion-MNIST for comparison as shown in Table 5.

Original: We follow the settings in Piratla et al.
Revised: We follow the settings in Piratla et al.

Original: (2020) and use ResNet-18 as the backbone.
Revised: (2020) and use ResNet-18 as the backbone.

Original: [Table]

Original: The conclusion is similar to that in PACS.
Revised: The conclusion is similar to that in PACS.

Original: Our method achieves better performance than both the non-adaptive domain generalization methods and adaptation methods.
Revised: Our method achieves better performance than both the non-adaptive domain generalization methods and adaptation methods.

Original: In Table 6 we directly evaluate using the variational  labels that are generated  at test-time.
Revised: [Table] [Table] Extra ablations We also provide extra ablation studies on the pseudo label generation and adaptation with our variational pseudo labels. We directly make predictions using the variational pseudo labels that are generated by sampling from the pseudo label distributions at test time.

Original: We observe that there is a difference empirically.
Revised: As shown in Table 6, the predictions based on the pseudo-label distributions are better than the ERM baseline, demonstrating that our variational pseudo labels are better than the original pseudo labels. Moreover, after adapting the model parameters by our variational pseudo labels, the performance further improves obviously [Table]

Revised: [Table]

Original: [Table]
Revised: [Table]

Revised: Time cost analyses.

Original: [Table]
Revised: We also provide the time cost of our method in both the training (Table 7) and inference stage (Table 8). As we utilize the meta-learning strategy to learn the ability to handle domain shifts during training, the time cost during training is larger than the ERM baseline. Moreover, compared with the ERM baseline, our variational pseudo-label learning and meta-learning framework only introduce a few parameters. Since the meta-learning strategy only complicates the training process, our method has similar runtime in inference compared with the other test-time adaptation methods, e

Original: [Table]
Revised: [Table]

Revised: g., Tent (Wang et al., 2021) and source-free domain adaptation methods, e

Original: [Table]
Revised: [Table]

Revised: g.

Revised: SHOT (Liang et al., 2020).