Variational Continual Test-Time Adaptation

Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-10 05:28:11

Abstract: Continual Test-Time Adaptation (CTTA) task investigates effective domain adaptation under the scenario of continuous domain shifts during testing time. 
Due to the utilization of solely unlabeled samples, there exists significant uncertainty in model updates, leading CTTA to encounter severe error accumulation issues.
In this paper, we introduce VCoTTA, a variational Bayesian approach to measure uncertainties in CTTA. 
At the source stage, we transform a pre-trained deterministic model into a Bayesian Neural Network (BNN) via a variational warm-up strategy, injecting uncertainties into the model. 
During the testing time, we employ a mean-teacher update strategy using variational inference for the student model and exponential moving average for the teacher model. 
Our novel approach updates the student model by combining priors from both the source and teacher models. 
The evidence lower bound is formulated as the cross-entropy between the student and teacher models, along with the Kullback-Leibler (KL) divergence of the prior mixture. 
Experimental results on three datasets demonstrate the method's effectiveness in mitigating error accumulation within the CTTA framework.

Main paper: 
 introduction: 
 Continual Test-Time Adaptation (CTTA) [51] aims to enable a model to accommodate a sequence of distinct distribution shifts during the testing time, making it applicable to various risk-sensitive applications in open environments, such as autonomous driving and medical imaging. However, realworld non-stationary test data exhibit high uncertainty in their temporal dynamics [23] , presenting challenges related to error accumulation [51] . Previous CTTA studies rely on methods that enforce prediction confidence, such as entropy minimization. However, these approaches often lead to predictions that are overly confident and less well-calibrated, thus limiting the model's ability to quantify risks during predictions. The reliable estimation of uncertainty becomes particularly crucial in the context of continual distribution shift [40] . It is meaningful to design a model capable of encoding the uncertainty associated with temporal dynamics and effectively handling distribution shifts. The objective of this paper is to devise a CTTA procedure that not only enhances predictive accuracy under distribution shifts but also provides reliable uncertainty estimates.To address the above problem, we refer to the Bayesian Inference (BI) [1] , which retains a distribution over model parameters that indicates the plausibility of different settings given the observed data, and it has been witnessed as effective in traditional continual learning tasks [38] . In Bayesian continual learning, the posterior in the last learning task is set to be the current prior which will be multiplied by the current likelihood. This kind of prior transmission is designed to reduce catastrophic forgetting in continual learning. However, this is not feasible in CTTA because unlabeled data may introduce unreliable prior. As shown in Fig. 1 , an unreliable prior may lead to a poor posterior, which may then propagate errors to the next inference, leading to the accumulation of errors. Figure 1 : In CTTA task, a BNN model is first trained on a source dataset, and then is used to adapt to updated with unreliable priors, which may result in error accumulations.Thus, we delve into the utilization of BI framework to evaluate model uncertainty in CTTA, aiming to mitigate the impact of unreliable priors and reduce the error propagation. To approximate the intractable likelihood in BI, we adopt to use online Variational Inference (VI) [49, 42] , and accordingly name our method Variational Continual Test-Time Adaptation (VCoTTA). At the source stage, we first transform a pretrained deterministic model, say CNN, into a Bayesian Neural Network (BNN) by a variational warm-up strategy, where the local reparameterization trick [27] is used to inject uncertainties into the source model. During the testing phase, we employ a mean-teacher update strategy, where the student model is updated via VI and the teacher model is updated by the exponential moving average. Specifically, for the update of the student model, we propose to use a mixture of priors from both the source and teacher models, then the Evidence Lower BOund (ELBO) becomes the cross-entropy between the student and teachers plus the KL divergence of the prior mixture. We demonstrate the effectiveness of the proposed method on three datasets, and the results show that the proposed method can mitigate the error accumulation in CTTA and obtain clear performance improvements.Our contributions are three-fold:(1) This paper develops VCoTTA, a simple yet general framework for continual test-time adaptation that leverages online VI within BNN.(2) We propose to transform an off-the-shelf model into a BNN via a variational warm-up strategy, which injects uncertainties into the model.(3) We build a mean-teacher structure for CTTA, and propose a strategy to blend the teacher's prior with the source's prior to mitigate unreliable prior problem.2 Related Work 
continual test-time adaptation: 
 Test-Time Adaptation (TTA) enables the model to dynamically adjust to the characteristics of the test data, i.e. target domain, in a source-free and online manner [25, 46, 50] . Previous works have enhanced TTA performance through the designs of unsupervised loss [37, 58, 32, 9, 7, 17] . These endeavours primarily focus on enhancing adaptation within a fixed target domain, representing a single-domain TTA setup, where models adapt to a specific target domain and then reset to their original pretrained state with the source domain, prepared for the next target domain adaptation.Recently, CTTA [51] has been introduced to tackle TTA within a continuously changing target domain, involving long-term adaptation. This configuration often grapples with the challenge of error accumulation [47, 51] . Specifically, prolonged exposure to unsupervised loss from unlabeled test data during long-term adaptation may result in significant error accumulation. Additionally, as the model is intent on learning new knowledge, it is prone to forgetting source knowledge, which poses challenges when accurately classifying test samples similar to the source distribution.To solve the two challenges, the majority of the existing methods focus on improving the confidence of the source model during the testing phase. These methods employ the mean-teacher architecture [47] to mitigate error accumulation, where the student learns to align with the teacher and the teacher updates via moving average with the student. As to the challenge of forgetting source knowledge, some methods adopt augmentation-averaged predictions [51, 2, 11, 55] for the teacher model, strengthening the teacher's confidence to reduce the influence from highly out-of-distribution samples.Some methods, such as [11, 6] , propose to adopt the contrastive loss to maintain the already learnt semantic information. Some methods believe that the source model is more reliable, thus they are designed to restore the source parameters [51, 2] . Though the above methods keep the model from confusion of vague pseudo labels, they may suffer from overly confident predictions that are less calibrated. To mitigate this issue, it is helpful to estimate the uncertainty in the neural network.Bayesian framework is natural to incorporate past knowledge and sequentially update the belief with new data [59] . The bulk of work on Bayesian deep learning has focused on scalable approximate inference methods. These methods include stochastic VI [22, 34] , dropout [16, 27] and Laplace approximation [41, 15] etc., and leveraging the stochastic gradient descent (SGD) trajectory, either for a deterministic approximation or sampling. In a BNN, we specify a prior p(θ) over the neural network parameters, and compute the posterior distribution over parameters conditioned on training data, p(θ|D) ∝ p(θ)p(D|θ). This procedure should give considerable advantages for reasoning about predictive uncertainty, which is especially relevant in the small-data setting.Crucially, when performing Bayesian inference, we need to choose a prior distribution that accurately reflects the prior beliefs about the model parameters before seeing any data [18, 14] . In conventional static machine learning, the most common choice for the prior distribution over the BNN weights is the simplest one: the isotropic Gaussian distribution. However, this choice has been proved indeed suboptimal for BNNs [14] . Recently, some studies estimate uncertainty in continual learning within a BNN framework, such as [38, 12, 13, 28] . They set the current prior to the previous posterior to mitigate catastrophic forgetting. However, the prior transmission is not reliable in the unsupervised CTTA task. Any prior mistakes will be enlarged by adaptation progress, manifesting error accumulation. To solve the unreliable prior problem, this paper proposes a prior mixture method based on VI. p(θ|DEQUATIONwhere p(θ|D 1:t ) denotes the posterior of sequential tasks on the learned parameter and p(D t |θ) is the likelihood of the current task. The current prior p t (θ) is regarded as the given knowledge. [38] proposes that this current prior can be the posterior learned in the last task, i.e., p t (θ) = p(θ|D 1:t-1 ),where the inference becomesEQUATIONThe detailed process can be shown in Appendix A.In contrast to continual learning, CTTA faces a sequence of learning tasks in test time without any label information, requiring the model to adapt to each novel domain sequentially. In this case, we assume that each domain is i.i.d. and the classes are separable following many unsupervised studies [36, 48, 5] , more details about the assumption can be seen in Appendix B.1. We use (3) Similarly, we can set the last posterior to be the current prior, i.e., p t (θ) = p(θ|U 1:t-1 ∪ D 0 ) and p 1 (θ) = p(θ|D 0 ). However, employing BI for adaptation on unlabeled testing data can result in untrustworthy posterior estimates. Therefore, during subsequent adaptation, the untrustworthy posterior automatically transform into unreliable priors, leading to error accumulation. In other words,U = {x n } N n=1 toFigure 2 : VCoTTA is built on mean-teacher structure, and conducts VI in CTTA using a mixture of teacher prior and source prior. The next teacher prior is updated by the exponential moving average. an unreliable prior p t (θ) will make the current posterior even less trustworthy. Moreover, the joint likelihood p(U t |θ) for t > 0 is intractable on unlabeled data.To make the BI feasible in CTTA task, in this paper, we transform the question to an easy-to-compute form. Referring to [20] , the unsupervised inference can be transformed intoEQUATIONwhere H denotes the conditional entropy and λ is a scalar hyperparameter to weigh the entropy term.This simple form reveals that the prior belief about the conditional entropy of labels is given by the inputs. The observation of the input U provides information on the drift of the input distribution, which can be used to update the belief over the learned parameters θ through Eq. ( 4). Consequently, this allows the utilization of unlabeled data for BI. More detailed derivations can be seen in Appendix B.2.In a BNN, the posterior distribution is often intractable and some approximation methods are required, even when calculating the initial posterior. In this paper, we leverage online VI, as it typically outperforms the other methods for complex models in the static setting [4] . VI defines a variational distribution q(θ) to approxmiate the posterior p(θ|U). The approximation process is as follows.EQUATION)where Q is the distribution searching space and Z t is the intractable normalizing hyperparameter.Thus, referring to the derivations in Appendix C, the ELBO is computed byEQUATIONOptimizing with Eq. ( 6) makes model adapt to domain shift. While VI offers a good framework for measuring uncertainty in CTTA, it is noteworthy that VI does not directly address the issue of unreliable priors. The error accumulation remains a significant concern.Despite this, the form of the ELBO in variational inference offers a pathway for mitigating the impact of unreliable priors. In Eq. ( 6), the entropy term may result in overly confident predictions that are less calibrated, while the KL term may be directly affected by an unreliable prior. In the following section, we will discuss how to solve the problems when computing the two terms.4 Adaptation and Inference in VCoTTA In the above section, we introduce the VI in CTTA but challenges remain, i.e., the unreliable prior.To mitigate the challenge in the entropy term, we adopt a Mean-Teacher (MT) structure [47] in the Bayesian inference process. MT is initially proposed in semi-supervised and unsupervised learning, where the teacher model guides the unlabeled data, helping the model generalize and improve performance with the utilization of large-scale unlabeled data.MT structure is composed of a student model and a teacher model, where the student model learns from the teacher and the teacher updates using Exponential Moving Average (EMA) [24] . In VI, the student is set to be the variational distribution q(θ), which is a Gaussian mean-field approximation for its simplicity. It is achieved by stacking the biases and weights of the network as follows.EQUATIONwhere d denotes each dimension of the parameter. The teacher model p(θ) (we use bar to distinguish the general prior) is also a Gaussian distribution. Thus, the student model is updated by aligning it with the teacher model through the use of a cross-entropy (CE) lossEQUATIONIn our implementation, we also try to use Symmetric Cross-Entropy (SCE) [53] in CTTA,EQUATIONSCE balances the gradient for high and low confidence, benefiting the unsupervised learning. 
kl term: mixture-of-gaussian prior: 
 For the KL term, to reduce the impact of unreliable prior, we propose a mixing-up approach to combining the teacher and source prior adaptatively. The source prior is warmed up upon the pretrained deterministic model p 1 (θ) = p(θ|D 0 ) (see Sec. 4.3.1). The teacher model pt (θ) is updated by EMA (see Sec. 4.3.3). We assume that the prior should be the mixture of the two Gaussian priors. Using only the source prior, the adaptation is limited. While using only the teacher prior, the prior is prone to be unreliable.We use the mean entropy derived from a given serious data augmentation to represent the confidence of the two prior models, and mix up the two priors with a modulating factorEQUATIONwhere I denotes augmentation types. θ 0 and θ are the parameters of the source model and the teacher model. τ means the temperature factor. Thus, as shown in Fig. 3 (b), the current prior p t (θ) is set to the mixture of priors asEQUATIONIn the VI, we use the upper bound to update the KL term [31] (see Appendix D.1) for simplicity,EQUATIONFurthermore, we also improve the teacher-student alignment in the entropy term (see Eq. ( 9)) by picking up the augmented logits with a larger confidence than the raw data. That is, we replace the teacher log-likelihood log p(x|θ) byEQUATIONwhere, for brevity, we let p(x ′ i ) = p(x ′ i |θ) and p(x) = p(x)|θ) in short. f (•) is the confidence function. ϵ denotes the confidence margin and 1(•) is an indicator function. Eq. ( 13) can be regarded as a filter, meaning that for each sample, the reliable teacher is represented by the average of its augmentations with ϵ more confidence. In Appendix D.2, we prove that the proposed mixture-of-Gaussian is benifical to CTTA. In Appendix E.1, we discuss the influence of different ϵ.To obtain a source BNN, instead of training a model from scratch on the source data D 0 , we transform a pretrained deterministic CNN to a BNN by variational warm-up strategy. Specifically, we leverage the local reparameterization trick [27] to add stochastic parameters, and warm up the model:EQUATIONwhere p(θ) represents the prior distribution, say the pretrained deterministic model. Eq. ( 14) denotes a standard VI on the source data, and we optimize the ELBO to obtain the variational distribution [49] .By the variational warm-up, we can easily transform an off-the-shelf pretrained model into a BNN with a stochastic dynamic. The variational warm-up strategy is outlined in Algorithm 1.Algorithm 1 Variational warm-up 1: Input: Source data D 0 , pretrained model p 0 (θ) 2: Initialize prior distribution p(θ) with p 0 (θ) 3: Update p(θ|D 0 ) ≈ q 0 (θ) by p(θ) and D 0 using Eq. ( 14) 4: Output: Source prior p 1 (θ) = p(θ|D 0 )The warm-up strategy is a common approach in TTA and CTTA tasks to further build knowledge structure for the source model, such as [26, 45, 11, 8] . Some other methods may not use warm-up but still use the source data, such as [39] . The warm-up strategy uses the source data only before deploying the model to CTTA scenario, and it is regarded as a part of pretraining. All of these methods using source data are operationalized in source-free at test time and find it is beneficial to CTTA. We use the warm-up to inject the uncertainties into a given source model, i.e., turning an off-the-shelf pretrained CNN model into a pretrained BNN model. This is convenient to obtain a pretrained BNN, because the warm-up strategy uses only a few epochs. We offer more discussions and experiments on the proposed variational warm-up strategy in Appendix F.The student model q t (θ) is adapted by approximating using Eq. ( 5), and is optimized on:EQUATIONwhere p′ t is the current augmented teacher model in Eq. ( 13), and p 1 (θ) ≈ q 0 (θ), pt (θ) ≈ qt (θ).The KL term between two Gaussians can be computed in a closed form.The teacher model is updated using EMA. Let (µ, σ) and ( μ, σ) be the mean and standard deviation of the student and teacher model, respectively. At test time, the teacher model qt (θ) is updated byEQUATIONAlthough the std is not used in the cross entropy to compute the likelihood, the teacher prior distribution is important to adjust the student distribution via the KL term.At any time, CTTA model needs to predict and adapt to the unlabeled test data. In our VCoTTA, we also use the mixed prior to serve as the inference model. That is, for a test data point x, the model inference is represented byEQUATIONFor the data prediction, the model only uses the expectation to reduce the stochastic, but leverages stochastic dynamics in domain adaptation. for Test data x ∼ U t do 5:Model predict for x (Eq. ( 17))6:Update student model using x (Eq. ( 15)) 7:Update teacher model via EMA (Eq. ( 16)) See more details in Appendix G. The process is feasible for any test data without labels. Pretrained Model. Following previous studies [50, 51] , we adopt pretrained WideResNet-28 [57] model for CIFAR10to-CIFAR10C, pretrained ResNeXt-29 [54] for CIFAR100-to-CIFAR100C, and standard pretrained ResNet-50 [21] for ImageNet-to-ImagenetC. Note in our VCoTTA [51] , we further warm up the pretrained model to obtain the stochastic dynamics for each dataset. Similar to CoTTA, we update all the trainable parameters in all experiments. The augmentation number is set to 32 for all compared methods that use the augmentation strategy. 
methods to be compared: 
 We compare our VCoTTA with multiple state-of-the-art (SOTA) methods. SOURCE denotes the baseline pretrained model without any adaptation. BN [30, 43] keeps the network parameters frozen, but only updates Batch Normalization. TENT [50] updates via Shannon entropy for unlabeled test data. CoTTA [51] builds the MT structure and uses randomly restoring parameters to the source model. SATA [6] modifies the batch-norm affine parameters using source anchoring-based self-distillation to ensure the model incorporates knowledge of newly encountered domains while avoiding catastrophic forgetting. SWA [55] refines the pseudo-label learning process from the perspective of the instantaneous and long-term impact of noisy pseudo-labels. PETAL [2] tries to estimate the uncertainty in CTTA, which is similar to BNN, but it ignores the unreliable prior problem.All compared methods adopt the same backbone, pretrained model and hyperparameters. We show the major comparisons with the SOTA methods in Tables 1, 2 and 3. We have the following observations. First, no adaptation at the test time (SOURCE) suffers from serious domain shift, whichshows the necessity of the CTTA. Second, traditional TTA methods that ignore the continual shift in test time perform poorly such as TENT and BN. We also find that simple Shannon entropy is effective in the first several domain shifts, especially in complex 1,000-classes ImageNetC, but shows significant performance drops in the following shifts. Third, the mean-teacher structure is very useful in CTTA, such as COTTA and PETAL, which means that the pseudo-label is useful in domain shift.In the previous method, the error accumulation leads to the unreliable pseudo labels, then the model may get more negative transfers in CTTA along the timeline. The proposed VCOTTA outperforms other methods on all the three datasets, such as 13.1% vs. 15.3% (SWA) on CIFAR10C, 28.4% vs. 30.0% (SWA) on CIFAR100C and 64.2% vs. 66.7% (COTTA) on ImageNetC. We hold the opinion that the prior will inevitably drift in CTTA, but VCOTTA slows down the process via the prior mixture. We also find that the superiority is more obvious in the early adaptation, which may be influenced by the different corruption orders. We analyze the order problem in Appendix H.We evaluate the two components in Table 4 , i.e., the Variational Warm-Up (VWU) and the Symmetric Cross-Entropy (SCE) via ablation. The ablation results show that the two components are both important for VCOTTA. First, the VWU is used to inject stochastic dynamics into an off-the-shelf pretrained model. Without the VWU, the performance of VCOTTA drops to 18.4% from 13.9% on CIFAR10C, 31.5% from 28.8% on CIFAR100C and 68.1% from 64.2% on ImageNetC. Also, the SCE can further improve the performance on CIFAR10C and CIFAR100C, because SCE balances the gradient for high and low confidence predictions. We also find that SCE is ineffective for complex ImageNetC, and the reason may be the class sensitivity imbalance, causing the model to lean more towards one direction during optimization. In Sec. 4.2, we introduce a Gaussian mixture strategy, where the current prior is approximated as the weighted sum of the source prior and the teacher prior. The weights are determined by computing the entropy over multiple augmentations of two models. To assess the effectiveness of these weights, we compare them with three naive weighting configurations: using only the source model, using only the teacher model, and a simple average with equal weights for both models. The results, as presented in Table 5 , reveal that relying solely on the source model or the teacher model (i.e., weighting with (1, 0) and (0, 1)) results in suboptimal performance. Additionally, naive weighting with equal contributions from both models (i.e., (0.5, 0.5)) proves ineffective for CTTA due to the inherent uncertainty in both models. In contrast, the proposed adaptive weights for the Gaussian mixture in CTTA demonstrate its effectiveness. This underscores the significance of striking a balance between the two prior models in an unsupervised environment. The trade-off implies the need to discern when the source model's knowledge is more applicable and when the teacher model's shifting knowledge takes precedence.To evaluate the uncertainty estimation, we use negative loglikelihood (NLL) and Brier Score (BS) [3] .Both NLL and BS are proper scoring rules [19] , and they are minimized if and only if the predicted distribution becomes identical to the actual distribution:NLL = -E (x,y)∈D test log(p(y|x, θ)), BS = E (x,y)∈D test (p(y|x, θ) -Onehot(y)) 2 ,where D test denotes the test set, i.e., the unsupervised test dataset U with labels. We evaluate NLL and BS with a severity level of 5 for all corruption types, and the compared results with SOTAs are shown in Table 6 . We have the following observations. First, most methods suffer from low confidence in terms of NLL and BS because of the drift priors, where the model is unreliable gradually, and the error accumulation makes the model perform poorly. Our approach outperforms most other approaches in terms of NLL and BS, demonstrating the superiority in improving uncertainty estimation. We also find that PETAL [2] shows good NLL and BS, because PETAL forces the prediction over-confident to unreliable priors, thus PETAL shows unsatisfactory results on adaptation accuracy, such as 31.5% vs. 28.4% (Ours) on CIFAR100C. We also show gradual corruption results instead of constant severity in the major comparison, and the results are reported in Table 7 . Specifically, each corruption adopts the gradual changing sequence:1 → 2 → 3 → 4 → 5 → 4 → 3 → 2 → 1,where the severity level is the lowest 1 when corruption type changes, therefore, the type change is gradual. The distribution shift within each type is also gradual. Under this situation, our VCoTTA also outperforms other methods, such as 8.9% vs. 10.5%(PETAL) on CIFAR10C, and 24.4% vs. 26.3% (COTTA) on CIFAR100C. The results show that the proposed VCOTTA based on BNN is also effective when the distribution change is uncertain.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: We provide the anonymous code link.Guidelines:• The answer NA means that paper does not include experiments requiring code.• Please see the NeurIPS code and data submission guidelines (https://nips.cc/ public/guides/CodeSubmissionPolicy) for more details.• While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).• The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //nips.cc/public/guides/CodeSubmissionPolicy) for more details.• The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.• The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.• At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).• Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
conclusion and limitation: 
 Conclusion: In this paper, we proposed a variational Bayesian inference approach, termed VCoTTA, to estimate uncertainties in CTTA. At the pretrained stage, we first transformed an off-the-shelf pretrained deterministic CNN into a BNN using a variational warm-up strategy, thereby injecting uncertainty into the source model. At the test time, we implemented a mean-teacher update strategy, where the student model is updated via variational inference, while the teacher model is refined by the exponential moving average. Specifically, to update the student model, we proposed a novel approach that utilizes a mixture of priors from both the source and teacher models. Consequently, the ELBO can be formulated as the cross-entropy between the student and teacher models, combined with the KL divergence of the prior mixture. We demonstrated the effectiveness of the proposed method on three datasets, and the results show that the proposed method can mitigate the issue of unreliable prior within the CTTA framework.Limitation: The efficacy of the proposed method relies on injecting uncertainty into the model during the pre-training phase, which may be unavailable in scenarios where pretraining is already completed, and original data is inaccessible. Additionally, constructing and training BNN models are inherently more complex compared to CNNs, highlighting the importance of enhancing computational efficiency.The Gaussian mixture method relies on multiple data augmentations, which also incurs computational costs. Future endeavors could explore more efficient approaches for Gaussian mixture.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: We follow previous to set the experiments.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.• The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: We offer the 10 different task orders to reduce the influence of stochastic and provide the avg ± std in Appendix H.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)• The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean.• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.• For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).• If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: We provide the compute resources in Appendix K.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.• The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.• The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: We confirm that we conducted in the paper conform with the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.• The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA] Justification: Nor applicable. We study machine learning problem on public dataset such as CIFAR10.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.• The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.• If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: No such risks. 
 
 Appendix: A Bayesian Inference (BI) in Traditional CL and CTTAAs described in Sec. 3, we first illustrate the BI has been studied in traditional Continual Learning (CL) methods. In this section, we compare the BI in CL and CTTA in detail and show the differences with some related works. The comparison can be seen in Fig. 3 . For the CL, BI is conducted by the posterior propagation, that is, the prior of next task is equal to the current posterior. This is feasible in supervised CL, where the data label is provided. For the CTTA, the posterior is not trustworthy using only pseudo labels to adapt to a new domain. Thus, propagate the untrustworthy posterior to the next stage would make unreliable prior, which will result in error accumulation. In the proposed VCoTTA, we propose to solve the problem via enhancing the two terms in VI (see Sec. 4). Figure 3 : Bayesian inference comparison between continual learning and CTTA. We find the traditional prior transmission is infeasible in CTTA because of the unreliable prior from unlabeled data. In our method, we place CTTA in a mean-teacher structure, and design BI in CTTA using a mixture of teacher prior and source prior. The next teacher prior is updated by the exponential moving average.VCL [38] is a classic CL study that uses VI, our work is also inspired by VCL but has the following difference. We also find another recent work named PETAL [2] that estimates uncertainties in CTTA. The BI formulation is similar between PETAL and ours, which is derived from [20] , but PETAL use different method to conduct the inference: (1) PETAL only uses CNN and does not estimate the model uncertainties, while VCoTTA uses BNN to model the uncertainties during test time. (2) PETAL ignores the unreliable prior in CTTA, and follow the VCL setting that use the previous posterior as the current prior. (3) We conduct BI using variational inference while PETAL use SWAG [35] .SWAG has advantages in terms of computational efficiency and stability during training, especially in scenarios where computational resources are limited. However, SWAG might not handle unreliable priors as effectively as VI since it doesn't explicitly model the posterior distribution. (4) We have compared with PETAL in our experiment (see Tables 1, 2 , 3), and our method outperforms PETAL on all datasets.In our method, we use the conditional entropy to alternate the intractable computing of likelihood.Note that the use of entropy in unsupervised scenario needs to satisfy the class-separable assumption.In fact, unlabeled data do not convey category information but still carry information. Miller and Uyar [36] theoretically proved that utilizing unlabeled samples to train classifiers can improve classification performance if there is a connection between the target and sample distributions.It is a common practice in unsupervised/semi-supervised learning to establish the relationship between unlabeled data and the target by making some reasonable assumptions to obtain categoryrelevant information from unlabeled data. Common assumptions include the Smoothness assumption,Cluster assumption, Manifold assumption, Low-density separation assumption, etc. For example, the well-known clustering-based methods utilize the cluster assumption to generate pseudo-labels for unsupervised learning [48] . Caron et al. [5] assumes that "the model trained on labeled data will produce high uncertainty estimation for unseen data" in domain adaptation tasks to benefit the classifier from unlabeled data lacking category information.Bengio et al. in [20] proposed the conditional entropy and point out that "These studies conclude that the (asymptotic) information content of unlabeled examples decreases as classes overlap. Thus, the assumption that classes are well separated is sensible if we expect to take advantage of unlabeled examples." This assumption has been applied to many studies, for example in [29, 33, 60, 2] . In the CTTA task of this paper, as the task progresses, the domain shifts, but the categories in the task remain unchanged. Therefore, under the assumption that unlabeled data contains information, we can reasonably continue to use conditional entropy in the current scenario. To sum up, whether in unsupervised TTA or in the Bayesian field, this assumption is not difficult to achieve or has never been applied. We can quite naturally continue to use this assumption in the context of this paper.The goal of CTTA is to learn a posterior distribution p(θ|U 1:T ∪ D 0 ) from a source dataset D 0 , and a sequence of unlabeled test data from U 1 to U T . Following [60] , assuming we have multiple input-generating distributions that the source dataset D 0 is drawn from a distribution ϕ, and φt specifies the shifted of the t-th unlabeled test dataset which we aim to adapt to. Let the parameters of the model be θ,then following the semi-supervised learning framework [20] , we incorporate all input-generating distributions into the belief over the model parameters θ as followsEQUATIONwhere the inputs X are sampled i. Following [60] , we can empirically use a point estimation to get a plug-in Bayesian approach to approximate the above formula:EQUATION)To make the formula feasible to CTTA, that is, no source data is available at the test time, we set λ 0 = 0. And the source knowledge can be represented by p(θ|D 0 ) ∝ p(θ) ∀x,y∈D0 p(y|x, θ).Thus, for the t-th test domain, the Bayesian inference in CTTA can be represented as follows:EQUATIONwhereH(U t |θ) = 1 |U t | ∀x∈Ut H(Y |x, θ)and the above formula can be rewritten in simplicity asEQUATIONwhich specifies the Bayesian inference process on continuously arriving unlabeled data in CTTA.We built VI for CTTA in Sec. 3, where we initialize a variational distribution q(θ) to approximate the real posterior. For the test domain t, we optimize the variational distribution as follows:EQUATION)where Q is the distribution searching space, and p t (θ) is the current prior.Following the definition of KL divergence and the standard derivation of the Evidence Lower BOund (ELBO) is as the following formulas. Specifically, the KL divergence is expanded asEQUATIONwhere the first constant term can be reduced in the optimization. Thus, we can optimize the variational distribution via the ELBO:EQUATION)In our case, the former entropy term can be more effectively replaced by the cross entropy or symmetric cross entropy (SCE) between the student model and the teacher model in a mean-teacher architecture (see Sec. 4.1). For the latter KL term, we can substitute a variational approximation that we deem closest to the current-stage prior p t (θ) into the KL divergence. When the prior is a multivariate Gaussian distribution, this term can be computed in closed form asEQUATION)where Σ = diag(σ 2 ), k represents the dimensionality of the distributions, tr(•) denotes the trace of a matrix, and det(•) stands for the determinant of a matrix. For the case that the prior is a mixture of Gaussian distributions, we can refer to the next section to get its upper bound.We refer to the lemma that was stated for the mixture of Gaussian in [44] . The KL divergence between two mixture distributions p =EQUATIONwhereα = (α 1 , α 2 , • • • , α k ) and α ′ = (α ′ 1 , α ′ 2 , • • • , α ′ k )are the weights of the mixture components.i p i / k j=1 α j p j = α ′ i p ′ i / k j=1 α ′ j p ′ j for all i.Using the log-sum inequality [10] , we haveKL( k i=1 α i p i ∥ k i=1 α i p ′ i ) = k i=1 α i p i log k i=1 α i p i k i=1 α i p ′ i ≤ k i=1 α i p i log α i p i α i p ′ i = k i=1 α i p i log α i α ′ i + p i log p i p ′ i = KL(α ∥ α ′ ) + k i=1 α i KL(p i ∥ p ′ i ).In our algorithm, q(θ) is set to be a mixture of Gaussian distributions, i.e., p t (θ) = α • p 1 (θ) + (1 - α) • pt (θ).In the above inequality, let q(θ) = k i=1 α i q(θ), we can get the upper bound of the KL divergence between q(θ) and p t (θ):EQUATION)So the lower bound (24) can be redefined asEQUATIONThen, we have obtained a lower bound that can be optimized through closed-form calculations as the source prior distribution q 0 (θ) and the teacher prior distribution qt (θ) are multivariate Gaussian distributions, which means we can also optimize L ′ with Eq. ( 25).In this subsection, we illustrate why the mixture of Gaussian prior are beneficial to CTTA. First of all, we can start from defining what is a better distribution for CTTA. Assume there exists an ideal prior distribution pt , which effectively represents the distribution of the model after learning all past knowledge, including that from the source and unlabeled datasets. Then we can use the difference between a distribution and the ideal distribution pt (here we use KL divergence) to measure the goodness of a distribution, i.e., KL(•||p t ).Generally, neither the source prior p 1 (trained on labeled data) nor the adapted prior pt (adapt on unlabeled data, being unreliable) can be completely consistent with pt . Considering that, as t increases, the difference between pt and pt will increase without an upper bound due to the error accumulation (since t is infinitely growing). The source prior p 1 cannot adapt to the unlabeled data, but it contains important information from the labeled data, and the ideal distribution cannot forget the source information too much, so we can assume that the difference between p 1 and pt is a constant, i.e., KL(p 1 ||p t ) < U , where U is a constant upper bound. Accordingly, it can be considered that mixing the source prior p 1 and the adapted prior pt in some way is beneficial for reducing KL(•||p t ).In our paper, we consider using a simple Gaussian mixture, i.e., p t = α t p 1 + (1 -α t )p t , where α is computed by Eq. (10) . It is easy to illustrate the benefits of this idea using the following inequality:EQUATIONIn Eq. ( 29), if KL(p t ||p t ) ≥ U , which can be satisfied as mentioned above, then we haveKL(p t ||p t ) ≤ KL(p t ||p t ),This indicates that the mixed distribution p t is closer to the ideal distribution pt than the adapted prior pt . A similar idea can be found in the stochatic restoration in CoTTA [51] , where the author randomly restore parts of parameters of the current model into the parameters of source model.In our method, we use the standard augmentation following CoTTA [51] . In this subsection, we analyze the some characteristics via experiments.First, we analyze the margin ϵ in Eq. ( 13). We experimentally validate different margins with more choices. Experimental results are shown in Tables 8 . The results indicate that different datasets may require different margins to control confidence. Moreover, Eq. ( 13) signifies that the reliable teacher likelihood is represented by the mean of its augmentations with ϵ more confidence than the teacher itself. Tables 8 illustrates the selection of ϵ in our approach on CIFAR10C, CIFAR100Cand ImageNetC. Note that when ϵ = -1, it means no margin is used and the method will use all augmentated samples, i.e., without using Eq. ( 13). The results show that the proposed margin can effectively filter out unreliable augmented samples and achieve a better teacher log-likelihood. In our method, we also use augmentation to enhance the confidence. We then evaluate the the number of augmentation in Eq. (10) . The results can be seen in Table 9 , and shows that increasing the number of augmentations can enhance effectiveness, but this hyperparameter ceases to have a significant impact after reaching 32. We have discussed the Variational Warm-Up (VWU) strategy in Sec. 4.3.1, and explain that the warm-up strategy is a common practice in TTA and CTTA. In this section, we further discuss some attributes of the proposed variational warm-up strategy.In our method, the VWU strategy is used to turn an off-the-shelf CNN to a pretrained BNN. The advantage of this approach is that pretrained CNNs are readily available (e.g., directly leveraging official models in PyTorch), while pretrained BNNs are challenging to obtain, especially for largescale datasets. Moreover, training BNNs is more difficult compared to training CNNs. Therefore, constructing BNN pretrained models based on existing CNN pretrained models is a feasible approach.Additionally, we find that such a warm-up strategy requires only a few epochs to achieve satisfactory results. To validate the characteristics of the proposed VWU strategy, we designed the following experiments.F.1 Warm-up on CNN vs. Directly Pretraining BNN First, we conducted experiments to compare the performance of obtaining pretrained BNN models using the warm-up approach versus directly training the source model with BNN. We pretrain the BNN also use VI as describing in Sec. 4.3.1. The results can be seen in Table 10 . As we can see, the results are at the same level, for example VI pretraining is with 13.2% error rate while the proposed VWU achieves 13.1% on CIFAR10C. However, if we direct turn a pretrained CNN to a BNN by adding random stochastic parameters, without warm-up strategy, the results drop to 17.1%. This shows that VWU is a feasible strategy to obtain a pretrained BNN. In our implementation, we employ only a limited number of epochs for variational warm-up, say 5 epochs. This is due to the fact that the pretrained model fits well in CNN, thus requiring minimal adjustments to the mean of BNN. Additionally, the standard deviation (std) is initialized to be small.Consequently, only a small number of iterations are necessary to update the BNN, and the step size is also kept small. Experimentation on the epoch number of variational warm-up reveals that keeping increasing epochs ( > 5) will diminishes performance, as shown in Fig. 5 . In this section, we show the algorithmic workflow utilizing variational approximation in VCoTTA. where we use the pretrained deterministic model p 0 (θ) as the prior distribution.When the domain shift: Then, at the beginning of the test time, we set the prior in task t asp t (θ) = α • p 1 (θ) + (1 -α)• pt (θ) and variational approximation, where p 1 (θ) ≈ q 0 (θ) and pt (θ) ≈ qt (θ). For qt (θ), which means the real-time posterior probability of the teacher model for the t-th test domain, is constantly updated by q t (θ) via EMA (see Sec. 4.3.3) during the test phase.Note that we do not have qt (θ) for the first update in the t-th phase. In fact, we use q t-1 (θ) construct the prior, thus we havep t (θ) ≈ α • p 1 (θ) + (1 -α) • q t-1 (θ). This is the variational distribution that should be used to approximate the prior in the absence of a teacher model in the first step, as well as the approximation that should be used when not employing the MT architecture. Note that the process is not required to inform the model that the domain produces a shift.During the testing time of a domain: With the approximation to p t (θ) and analysis from Appendix B.2, we get q t (θ) for student model at the test domain t as follows:EQUATIONwhich means, we can recursively derive p t+1 (θ) and the following variational distributions, thereby achieving the goal of VCoTTA.As we discuss in the major comparisons (see Sec 5.3), the performance may be affected by the corruption order. To provide a more comprehensive evaluation of the matter of the order, we conduct 10 different orders from Sec 5.3, and show the average performance of all compared methods.10 independent random orders of corruption are all under the severity level of 5. The results are shown in Table 11 . We find that the order of corruption is minor on simple datasets such as CIFAR10C and CIFAR100C, but small std on difficult datasets such as ImageNetC. The proposed VCOTTA outperforms other methods on the average error of CIFAR10C and CIFAR100C under 10 different corruption orders, which shows the effectiveness of the prior calibration in CTTA. Moreover, VCOTTA has comparable results with PETAL on ImageNetC, but smaller std over 10 orders, whichshows the robustness of the proposed method. In the real-world scenario, the testing domain may reappear in the future. We evaluate the test conditions continually 10 times to evaluate the long-term adaptation performance on CIFAR10C.That is, the test data will be re-inference and re-adapt for 9 more turns under severity 5. Full results can be found in Fig. 6 . The results show that most compared methods obtain performance improvement in the first several loops, but suffer from performance drop in the following loops. This means that the model drift can be even useful in early loops, but the drift becomes hard because of the unreliable prior. The results also indicate that our method outperforms others in this long-term adaptation situation and has only small performance drops. CTTA does operate in an online setting, where all testing data is used only once. However, the current focus of CTTA research primarily revolves around batch-mode online settings, with batch sizes typically set to 200 in our experiments like other SOTAs. In CTTA, strict online learning settings where each data point is processed individually are under-researched. In fact, our method can be applied in scenarios with online learning or small batch sizes. However, it's important to note that the batch normalization (BN) layers is disabled when the batch size is 1. We experimented with batch size of 1 on CIFAR10C, and compare the results with some baseline methods. The comparison results are shown in Table 12 . The results show that small batch size in CTTA makes worse performance.We believe this is because a small batch size amplifies the uncertainty in model training. We implement our method using a single RTX-4090 GPU card. We provide the memory and time cost in Table 13 . Our proposed VCoTTA method does not offer an advantage in terms of memory usage. This is because in the BNN framework, additional standard deviations are required for implementing local reparameterization tricks. However, during the testing phase, this does not significantly impact the efficiency of the model. This is because during testing, only the student model employs variational inference, which requires uncertainty parameters. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.• Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.• While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 


Decision: This paper presents a variational Bayesian approach to the problem of continual test-time adaptation (TTA). The proposed approach is inspired by recent works on Bayesian TTA [60], with additional measures to handle the issue of catastrophic forgetting in continual settings. These additional measures are based on recent works on continual TTA [51].

The reviewers appreciate the work for taking a Bayesian approach for continual TTA. However, these were also concerns regarding experiments, e.g., no experiments on other tasks such as segmentation, lack of standard errors in the reported results, and lack of comparison with recent SOTA methods. For the last point, in the rebuttal, although the authors did provide some preliminary numbers (on one dataset, and only for accuracy metric) comparing with EcoTTA, a more detailed experimental comparison is desirable.

In the end, the paper remained somewhat on the borderline. The authors' responses (to the reviewers as well as private comments to the AC) were considered but concerns still remain. The Bayesian setup by itself is not totally a novel idea for TTA (though not for the continual setting, [60] did consider Bayesian TTA, whose formulation is somewhat similar to the proposed method, and then PETAL [2] also had a probabilistic approach to continual TTA) and, given that there is extensive amount of work on continual TTA, the paper also fell short on reporting more extensive comparisons with SOTA methods. I think with the additional comparions with such methods (on multiple datasets, and for classification as well as segmentation, as is routinely done in most continual TTA papers nowadays), it would be a much stronger paper.

The authors are advised to incorporate these suggestions and resubmit the work to another venue.

Reviews: review: paper_summary: paper_summary: The paper presents a variational Bayesian approach to handle uncertainties in continual test-time adaptation (CTTA). The source pretrained model is made Bayesian by variational warm and a mean-teacher update strategy is used at test time. To avoid drift due to uncertainty of priors using only unlabeled data at test time, the paper proposed to update the student model by combining priors from both the source and teacher models. The evidence lower bound is formulated as the cross-entropy between the student and teacher models, along with the Kullback-Leibler (KL) divergence of the prior mixture. Experimental results on three datasets demonstrate the method’s effectiveness in mitigating error accumulation within the CTTA framework.
strength_weakness: strength_weakness: - While Bayesian approach is nice in principle, it can be computationally demanding and offer little benefit in practice. Most of the existing CTTA methods are computationally and memory efficient, whereas this method present an opposite end of the spectrum. While the reported results are impressive, it is unclear to me why the proposed method is superior to other SOTA methods.
- The hyperparameter selection process is not addressed in the paper, which is critical in TTA where all hyperparameters should be predetermined before data access. How are they chosen?
questions: questions: - What is the rough analysis of the space and time complexity of the proposed approach, separate for off-line and online phases? There is only one incomplete empirical comparison in the appendix, which leave me curious how the space and time cost change in different settings (e.g., batch size, training set size, model size, etc).

- In the experiments, can resetting to source simply solve the drift issue and achieve much better results than the naive results in the tables?

- For the mixture rate alpha, what is the best result with a non-adaptive alpha?
limitations: limitations: Limitations are mentioned in the paper, albeit very brief. Overall, the proposed method is more complex and demanding (such as requiring a pretrained probabilistic model or source data) for TTA applications. Perhaps the proposed approach may work even better with UDA or other CL scenarios with (partially available) target labels?
paper_summary: paper_summary: The paper proposes a method to continually adapt a pre-trained classifier to an unlabeled stream of test data. They address the problem of continual test-time adaptation through the lens of Bayesian deep learning. Their method consists of three main components: (1) a variational warm-up strategy to turn any source model into a Bayesian Neural Network, (2) a mixing strategy between the source model and the last posterior to leverage the trade-off between adaptation and forgetting, and (3) a modified entropy term that is symmetric and incorporates data augmentations. The authors compare their method on standard CIFAR-C and Imagenet-C datasets to a set of TTA baselines. The paper also includes ablation studies on the components and an evaluation of uncertainty estimates.
strength_weakness: strength_weakness: The paper presents strong evidence in support of the proposed method. However, it is left unclear to me why the method performs so much better than previous approaches.

- The method consists of a range of specific components. However, in some cases, the specific design of the components is not clearly motivated. In particular, equations 10 and 13 lack supporting citations or explanations. Why have exactly these formulations been chosen?
- I’d like to get more clarity on the difference between this paper and the original CoTTA work, as it seems to me there are certain components in common (e.g., student-teacher approach, EMA). More precisely, could you please highlight the difference in the update equations between the two papers? My understanding is that adding the VI framework notably changes (i) the optimization objective by adding the KL term (instead of solely minimizing entropy) and (ii) the predictions by marginalizing out the model parameters. Where else does the VI framework contribute to differences?
questions: questions: - Recent work has explored the degradation of CTTA methods after a long adaptation period [1]. Given that the paper argues that the mixed prior between the adapted and source model prevents forgetting, it would be interesting to see for how long the adaptation can be successful. Is there any experimental evidence on the robustness over time of the adaptation method? See [1] for an example of a suitable benchmark dataset.
- What is the motivation behind Equation 13? In other words, why would we only want to have log-likelihood terms of “confident augmentations”? I would have assumed that having a large pool of augmentations and averaging them all would give more robust predictions. An ablation study on the design of this term would also be insightful.
- Could the method suffer from mode averaging in Equation 11?
- Am I assuming correctly that only one gradient step is performed per test batch?

[1] Press, Ori, et al. "Rdumb: A simple approach that questions our progress in continual test-time adaptation." Advances in Neural Information Processing Systems 36 (2024).
limitations: limitations: The authors have listed limitations including computational efficiency and the need for access to the source data at adaptation time
paper_summary: paper_summary: This paper introduces VCoTTA, a novel variational Bayesian approach to address the Continual Test-Time Adaptation (CTTA) task, which focuses on effective domain adaptation during continuous domain shifts at test time. The authors' main contributions include a method to measure uncertainties in CTTA, addressing the issue of error accumulation due to the use of unlabeled samples. They propose transforming a pretrained deterministic model into a Bayesian Neural Network (BNN) using variational warm-up at the source stage, and employ a mean-teacher update strategy during test time. The approach updates the student model by combining priors from both source and teacher models, with the evidence lower bound formulated as the cross-entropy between student and teacher models, along with the Kullback-Leibler (KL) divergence of the prior mixture. Experimental results on three datasets demonstrate the method's effectiveness in mitigating error accumulation within the CTTA framework.
strength_weakness: strength_weakness: Regarding the computational overhead discussed in the paper, while it is noted that online Variational Inference is employed to make the approach computationally feasible, a detailed analysis of the computational costs associated with VCoTTA is absent. Table 13 presents a comparison of time and memory costs, but the source of these values is unclear. Could you specify which dataset was used for these measurements? Also, is it possible to clarify whether the time and memory comparisons pertain to training or testing phases?

The manuscript contains several typographical and grammatical errors that need to be addressed. Specifically, brackets are missing in Equation (5) and in the sentence following Equation (13). Could these omissions be corrected to prevent misinterpretation of the mathematical expressions and enhance the clarity of the paper?

There are multiple grammatical issues that require rectification. The sentence "MT is initially proposed in semi-supervised and unsupervised learning" is somewhat unclear. Could this be rephrased for better coherence? Additionally, the sentence "We use the mean entropy derived from a given *serious* data augmentation to represent the confidence of the two prior models, and mix up the two priors with a modulating factor" appears to contain a typographical error and could be better structured. Could these issues be addressed to improve the readability and accuracy of the text?

The heading for Section 5.7 seems to not accurately reflect the content discussed within. Could this heading be revised to more accurately convey the main topics or findings of the section, thereby ensuring clarity and relevance for the reader?

The explanation of how MT operates in semi-supervised and unsupervised learning settings appears incomplete and potentially misleading. The current statement, "where the teacher model guides the unlabeled data, helping the model generalize and improve performance with the utilization of large-scale unlabeled data," lacks specificity. Could you specify which model (teacher or student) benefits from this guidance and in what manner? Additionally, the phrase "where the teacher model guides the unlabeled data" seems incorrect. Could this be clarified or corrected to accurately reflect the operational dynamics of the MT framework?
questions: questions: Please provide answers to the questions/suggestions provided in the Weaknesses section.
limitations: limitations: The authors have adequately addressed the limitations in the paper.
paper_summary: paper_summary: The authors propose a variational continual adaptation method. Where a sequence of test-time domain adaptation problems are shown to a model. More specifically a labeled dataset is given as an initial dataset to learn from, and then afterwards a sequence of unlabelled datasets with domain shifts are presented to the model.

Using traditional variational continual learning methods will result in error accumulations in the posterior over parameters. So the authors propose a scheme that regularizes against the posterior learned from an initial source dataset. 

The authors propose do away with using sequential Bayesian inference. Instead, the authors use a mix of a source prior (learned using labeled data) and a teacher prior. The teacher prior is an EMA of the previous task’s posterior learned with variational inference. The method the authors propose is named VCoTTA.
strength_weakness: strength_weakness: Novelty:
* In terms of novelty I’m not convinced that there are important applications of continual test-time adaptation in the form of classification tasks derived from CIFAR10 datasets. I could be wrong, but some justification in the paper is required and some more realistic benchmarks would be nice.
* Maybe I have misunderstood the variational warm-up procedure. But using the MLE estimates to initialize the BNN mean parameters was done in VCL https://arxiv.org/abs/1710.10628 . So this is not a novel idea. Furthermore, there are better ways to initialize a BNN such as using Bayesian linear regression: https://proceedings.mlr.press/v97/rossi19a/rossi19a.pdf.

Clarity
* Why does the teacher model use EMA updates instead of using the inference variational posterior?
* Why is data augmentation an important component in VCoTTA (Sec 4.2)? This is suddenly presented in the paper without justification.

Notation:
* In Section 4.2, the title is a “Mixture-of-Gaussian prior”, but Eq 11 is an addition of two priors which are Gaussians by design, so this is a “scale-mixture prior” https://arxiv.org/pdf/1505.05424, not a mixture of Gaussians (https://www.inf.ed.ac.uk/teaching/courses/mlpr/2016/notes/w9b_mixture_models.pdf)?
* Confusing notation of the source prior: it is denoted as $p_0$ (Fig 2) and $p_1$ (Eq 11), this needs to be consistent.


Empirical weaknesses:
* No standard errors in the experimental results. So difficult to see which method outperforms another.
* Uncertainty estimation is not performed with standard methods like ECE or OOD detection like https://arxiv.org/abs/2102.06571. It is unclear to me whether the Brier Score estimates uncertainties.
questions: questions: Please see the weaknesses above. Additionally, there are some other minor questions mainly regarding the writing and the questions around implementation:
* Line 169: what is a “serious data augmentation”?
* Line 178: what is a confidence function?
* Eq 13: what is $x’$?
* Alg 1, line 2: how do you initialize the prior distribution $p(\theta)$ with the MLE weights from a pre-trained CNN? Do you set the mean variational means to the MLE values and initialize the variances to be some small initial value such as $10^-3$?
* On the first domain adaptation update of VCOTTA, how is $\bar{p}_1$ defined since we do not have an initial teacher model yet (Eq 11)?
limitations: limitations: There is a good discussion on the limitations of VCoTTA.

One limitation that is not discussed is in the effectiveness of (variational) Bayesian sequential inference methods. Weight space variational inference has been shown to be very difficult to do in practice, https://arxiv.org/abs/2301.01828. So weight space variational inference (without tricks like multi-head networks and coresets) might not be the best choice when wanting to remember a source distribution.

Comments: title: Response to rebuttal
comment: Thank you for very much for providing this overview. It addressed the concern I raised initially. I raised my score to 5.
Thank you for the additional results. These resolved some additional concerns I had about the paper. I am raising my score from 5 to 6.
Thank you for your further comments, and our responses are as follows.

**A brief description of the effective reason**: To illustrate why it is effective, we simplify the description of our key idea: The characteristics of BNN make it more effective in uncertain testing scenarios, but directly using BNNs in CTTA may lead to error accumulation due to unreliable priors. To address these unreliable priors, we propose the VCoTTA method, which enhances the performance of VI in CTTA tasks by enhancing the priors in a Mean-Teacher structure.

**Two other SOTA methods to compare**: Thank you for your kind suggestion. Due to space constraints in the manuscript, we choose some other SOTA methods for comparison such as PETAL (CVPR24), which is more related to our motivation. We show the comparison results on CIFAR10C with the two methods in the table below, and we will to include these comparisons in the revised manuscript.
  | Method | Avg. err |
  |---|---|
  | ECoTTA  | 16.8 |
  | Vida | 15.8 |
  | Ours  | 13.1 |
Thank you for your comments, we response to the difference of low-level distinctions between CoTTA and VCoTTA as follows.
  
1. **CoTTA Contributions and Equations**: First, let's review the three main *contributions of CoTTA* and then explain the *equation differences* between our approach and CoTTA in regard to the 3 contributions and 8 equations. Note that Eq. 1, 2 and 5 are the updates for student and teacher models.

	- *Weight-Averaged Pseudo-Labels*: This contribution is in fact the Mean-Teacher (MT) structure use in CTTA. We also adopt the MT structure like CoTTA’s design. The equation differences are:
		- CoTTA Eq. 1&5: Student update. The reviewer has mentioned this, and our further explanation is provided below (Response 2).
		- CoTTA Eq. 2: Teacher update. Both CoTTA and VCoTTA use EMA for updating, but VCoTTA is based on BNNs (Our Eq. 16), where the BNN is updated in Gaussian distribution.
	- *Augmentation-Averaged Pseudo-Labels* (CoTTA Eq. 3&4): CoTTA chooses to use either augmented or non-augmented teacher pseudo-labels based on whether the source model's confidence in the sample exceeds a certain threshold. In our method, we do not make such a judgement. For the augmentation, we first leverage data augmentation to calculate the mixing coefficient between the teacher prior and the source prior (Our Eq. 10), then enhance the teacher prior (Our Eq. 13) as we mentioned in the rebuttal.
	- *Stochastic Restoration* (CoTTA Eq. 6&7&8): CoTTA randomly restores part of student model parameters to the corresponding parameters in the source model with a certain probability to preserve the knowledge from the source. Our method does not employ this stochastic restoration strategy. Our mixture of priors (Our Eq. 11,12,15) may have similar form as CoTTA Eq. 8, but we do not restore student model using source model randomly but mix the teacher prior and source prior with an adaptive factor $\alpha$.

2. **Difference in Student loss**: The reviewer has pointed out the student loss difference in low-level. Compared to CoTTA, the main difference in student loss is the KL term at the end of our Eq.15. The KL term can be considered a regularization constraint on the model parameters. $q_t$ , $q_0$ and $\bar{q}_t$ represent the parameter distributions of the student model (to be optimized), source model, and augmented teacher model, respectively. In our method, all three distributions are Gaussian. The calculation of the KL term for Gaussian distributions can be referenced in Eq. 25 of our appendix, which can be computed directly in closed form. The meaning of the KL term is to provide constraints on the student model update when faced with unreliable priors, using both the source model and teacher model. These two constraints are controlled by the weights calculated in our Eq. 10.
  
Thank you to the reviewer for pointing out that we need to improve our method presentation. We will enhance the clarity of our descriptions in the paper to make the meaning of the formulas easier for readers to understand.
>**Weakness 2: Motivations of our design**
>

I appreciate the additional comments on Equations 10 and 13. I would like to see them included in the revised version. I believe that providing a more detailed explanation of how these design choices are motivated would strengthen the paper.

>**Weakness 3: Comparison with CoTTA**
>

Thank you for the response. I believe I understand the high-level differences between the CoTTA and VCoTTA papers as provided in your response. However, my question was aimed at the lower-level distinctions between the two approaches. Despite the high-level difference between the two (e.g., CNN vs. BNN perspective), I do believe there are many lower-level similarities. In my question, I’d like to get clarity about the low-level differences (see the example of the student loss below). I feel this has not yet been addressed.

> *(3) About the reviewer's understanding*: The objective the reviewer mentioned is the ELBO (Equation 6) of Variational Inference (VI) in CTTA, which is derived from the VI assumption. VI assumes that there exists a variational distribution q(θ) that approximates the true posterior p(θ|U). The approximation process can be represented by a KL divergence optimization (Equation 5). However, it is difficult to directly optimize the KL divergence, so the ELBO is an alternative for optimization. For prediction, the BNN can be reduced to a CNN because the variance is not used.
>

My description might have been misleading. I am referring to the student loss of VCoTTA (Equation 15) and comparing it to the student loss of CoTTA (Equation 1 in [1]). I think one can see an example of a lower-level difference there (i.e., the additional term in VCoTTA, $\alpha \times KL (q_t||q_0) + (1 − \alpha) \times KL (q_t|| \bar{q}_t)$).

I’d like to get a better overview of these kinds of differences (i.e., differences in the *equations*). I think this could provide the reader with a better understanding of why the method performs so well (weakness 1).

[1] Continual Test-Time Domain Adaptation, CVPR, 2022.

>**Question 1: Long adaptation period**
>
Thank you for pointing to the additional experiment. This answers my question.

>**Question 2: Motivation and ablation study on Equation 13**
>
Thank you for the clarifications. My question has been answered.
Thanks for taking the time to respond to my queries.

## BNN initialization

> The initialization used by VCL is "the prior $N(0, 1)$ and ... a very small initial variance (10^−6)", which is a direct initialization. However, VWU is different from the initialization method. The significant difference is that we require a complete pre-trained CNN, and the advantage of VWU can be seen in Appendix F. In Tab. 10, we have compared with common initialization and find that VWU is more convenient.

Yes, the prior is $N(0, 1)$, but the variational mean parameters are initialized by using a network with the same architecture pre-trained using MLE. This might not be stated explicitly, but is a feature of the implementation and a common feature of variational BNNs.

## Overall

I will raise my score, but still do not think this is ready for publication. I would like to see a more real-world scenario beyond classification problems to really show case TTA. I would also like to see standard errors by default on most if not all experiments.
Thank you for a detailed response to my questions. 
- The additional results are helpful to understand the complexity of the approach and the model selection better.
- The answer to why it's superior to SOTA is still circumstantial but I understand that it's not an easy question.
- Speaking of SOTA, it came to my attention that the experiments are missing comparisons with ViDA [1] and EcoTTA [2] which are frequently cited in other papers. Any reason why?


[1] Liu, Jiaming, et al. "Vida: Homeostatic visual domain adapter for continual test time adaptation." arXiv preprint arXiv:2306.04344 (2023).
[2] Song, Junha, et al. "Ecotta: Memory-efficient continual test-time adaptation via self-distilled regularization." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.