Title: Variational Continual Test-Time Adaptation
Venue: NeurIPS 2024
Accepted: no
Keywords: Continual test-time adaptation

Continual learning

Domain adaptation

Test-time adaptation
Abstract: Continual Test-Time Adaptation (CTTA) task investigates effective domain adaptation under the scenario of continuous domain shifts during testing time. 
Due to the utilization of solely unlabeled samples, there exists significant uncertainty in model updates, leading CTTA to encounter severe error accumulation issues.
In this paper, we introduce VCoTTA, a variational Bayesian approach to measure uncertainties in CTTA. 
At the source stage, we transform a pre-trained deterministic model into a Bayesian Neural Network (BNN) via a variational warm-up strategy, injecting uncertainties into the model. 
During the testing time, we employ a mean-teacher update strategy using variational inference for the student model and exponential moving average for the teacher model. 
Our novel approach updates the student model by combining priors from both the source and teacher models. 
The evidence lower bound is formulated as the cross-entropy between the student and teacher models, along with the Kullback-Leibler (KL) divergence of the prior mixture. 
Experimental results on three datasets demonstrate the method's effectiveness in mitigating error accumulation within the CTTA framework.

Abstract

Continual Test-Time Adaptation (CTTA) task investigates effective domain adaptation under the scenario of continuous domain shifts during testing time. Due to the utilization of solely unlabeled samples, there exists significant uncertainty in model updates, leading CTTA to encounter severe error accumulation issues. In this paper, we introduce VCoTTA, a variational Bayesian approach to measure uncertainties in CTTA. At the source stage, we transform a pretrained deterministic model into a Bayesian Neural Network (BNN) via a variational warm-up strategy, injecting uncertainties into the model. During the testing time, we employ a mean-teacher update strategy using variational inference for the student model and exponential moving average for the teacher model. Our novel approach updates the student model by combining priors from both the source and teacher models. The evidence lower bound is formulated as the cross-entropy between the student and teacher models, along with the Kullback-Leibler (KL) divergence of the prior mixture. Experimental results on three datasets demonstrate the method's effectiveness in mitigating error accumulation within the CTTA framework. Our code is anonymously available at https://anonymous.4open.science/r/vcotta-D2C3/ .

1 Introduction

Continual Test-Time Adaptation (CTTA) [51] aims to enable a model to accommodate a sequence of distinct distribution shifts during the testing time, making it applicable to various risk-sensitive applications in open environments, such as autonomous driving and medical imaging. However, realworld non-stationary test data exhibit high uncertainty in their temporal dynamics [23], presenting challenges related to error accumulation [51]. Previous CTTA studies rely on methods that enforce prediction confidence, such as entropy minimization. However, these approaches often lead to predictions that are overly confident and less well-calibrated, thus limiting the model's ability to quantify risks during predictions. The reliable estimation of uncertainty becomes particularly crucial in the context of continual distribution shift [40]. It is meaningful to design a model capable of encoding the uncertainty associated with temporal dynamics and effectively handling distribution shifts. The objective of this paper is to devise a CTTA procedure that not only enhances predictive accuracy under distribution shifts but also provides reliable uncertainty estimates.

To address the above problem, we refer to the Bayesian Inference (BI) [1], which retains a distribution 30 over model parameters that indicates the plausibility of different settings given the observed data, and 31 it has been witnessed as effective in traditional continual learning tasks [38]. In Bayesian continual 32 learning, the posterior in the last learning task is set to be the current prior which will be multiplied 33 by the current likelihood. This kind of prior transmission is designed to reduce catastrophic forgetting 34 in continual learning. However, this is not feasible in CTTA because unlabeled data may introduce 35 unreliable prior. As shown in Fig. 1, an unreliable prior may lead to a poor posterior, which may then 36 propagate errors to the next inference, leading to the accumulation of errors. 37

BNN

BNN

BNN

Source

BNN

Domain shift

Domain shift

Adaptation using

unreliable prior

Error

accumulation

...

Domain

shift

Train

Test

Thus, we delve into the utilization of BI framework to evaluate model uncertainty in CTTA, aiming 38 to mitigate the impact of unreliable priors and reduce the error propagation. To approximate the 39 intractable likelihood in BI, we adopt to use online Variational Inference (VI) [49, 42], and accordingly 40 name our method Variational Continual Test-Time Adaptation (VCoTTA). At the source stage, 41 we first transform a pretrained deterministic model, say CNN, into a Bayesian Neural Network 42 (BNN) by a variational warm-up strategy, where the local reparameterization trick [27] is used to 43 inject uncertainties into the source model. During the testing phase, we employ a mean-teacher 44 update strategy, where the student model is updated via VI and the teacher model is updated by 45 the exponential moving average. Specifically, for the update of the student model, we propose to 46 use a mixture of priors from both the source and teacher models, then the Evidence Lower BOund 47 (ELBO) becomes the cross-entropy between the student and teachers plus the KL divergence of the 48 prior mixture. We demonstrate the effectiveness of the proposed method on three datasets, and the 49 results show that the proposed method can mitigate the error accumulation in CTTA and obtain clear 50 performance improvements. 51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

Our contributions are three-fold:

This paper develops VCoTTA, a simple yet general framework for continual test-time adaptation that leverages online VI within BNN.

We propose to transform an off-the-shelf model into a BNN via a variational warm-up strategy, which injects uncertainties into the model.

We build a mean-teacher structure for CTTA, and propose a strategy to blend the teacher's prior with the source's prior to mitigate unreliable prior problem.

2 Related Work

2.1 Continual Test-Time Adaptation

Test-Time Adaptation (TTA) enables the model to dynamically adjust to the characteristics of the test data, i.e. target domain, in a source-free and online manner [25, 46, 50]. Previous works have enhanced TTA performance through the designs of unsupervised loss [37, 58, 32, 9, 7, 17]. These endeavours primarily focus on enhancing adaptation within a fixed target domain, representing a single-domain TTA setup, where models adapt to a specific target domain and then reset to their original pretrained state with the source domain, prepared for the next target domain adaptation.

Recently, CTTA [51] has been introduced to tackle TTA within a continuously changing target 67 domain, involving long-term adaptation. This configuration often grapples with the challenge of error 68 accumulation [47, 51]. Specifically, prolonged exposure to unsupervised loss from unlabeled test 69 data during long-term adaptation may result in significant error accumulation. Additionally, as the 70 model is intent on learning new knowledge, it is prone to forgetting source knowledge, which poses 71 challenges when accurately classifying test samples similar to the source distribution. 72

To solve the two challenges, the majority of the existing methods focus on improving the confidence of 73

the source model during the testing phase. These methods employ the mean-teacher architecture [47] 74

75

to mitigate error accumulation, where the student learns to align with the teacher and the teacher

updates via moving average with the student. As to the challenge of forgetting source knowledge, 76 some methods adopt augmentation-averaged predictions [51, 2, 11, 55] for the teacher model, 77 strengthening the teacher's confidence to reduce the influence from highly out-of-distribution samples. 78 Some methods, such as [11, 6], propose to adopt the contrastive loss to maintain the already learnt 79 semantic information. Some methods believe that the source model is more reliable, thus they are 80 designed to restore the source parameters [51, 2]. Though the above methods keep the model from 81 confusion of vague pseudo labels, they may suffer from overly confident predictions that are less 82 calibrated. To mitigate this issue, it is helpful to estimate the uncertainty in the neural network. 83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

2.2 Bayesian Neural Network

Bayesian framework is natural to incorporate past knowledge and sequentially update the belief with new data [59]. The bulk of work on Bayesian deep learning has focused on scalable approximate inference methods. These methods include stochastic VI [22, 34], dropout [16, 27] and Laplace approximation [41, 15] etc., and leveraging the stochastic gradient descent (SGD) trajectory, either for a deterministic approximation or sampling. In a BNN, we specify a prior p ( θ ) over the neural network parameters, and compute the posterior distribution over parameters conditioned on training data, p ( θ |D ) ∝ p ( θ ) p ( D| θ ) . This procedure should give considerable advantages for reasoning about predictive uncertainty, which is especially relevant in the small-data setting.

Crucially, when performing Bayesian inference, we need to choose a prior distribution that accurately reflects the prior beliefs about the model parameters before seeing any data [18, 14]. In conventional static machine learning, the most common choice for the prior distribution over the BNN weights is the simplest one: the isotropic Gaussian distribution. However, this choice has been proved indeed suboptimal for BNNs [14]. Recently, some studies estimate uncertainty in continual learning within a BNN framework, such as [38, 12, 13, 28]. They set the current prior to the previous posterior to mitigate catastrophic forgetting. However, the prior transmission is not reliable in the unsupervised CTTA task. Any prior mistakes will be enlarged by adaptation progress, manifesting error accumulation. To solve the unreliable prior problem, this paper proposes a prior mixture method based on VI.

3 Variational Inference in CTTA

We start from the supervised BI in typical continual learning, where the model aims to learn multiple classification tasks in sequence. Let D = { ( x n , y n ) } N n =1 be the training set, where x n and y n denotes the training sample and the corresponding class label. The task t is to learn a direct posterior approximation over the model parameter θ as follows.

where p ( θ |D 1: t ) denotes the posterior of sequential tasks on the learned parameter and p ( D t | θ ) is the likelihood of the current task. The current prior p t ( θ ) is regarded as the given knowledge. [38] proposes that this current prior can be the posterior learned in the last task, i.e. , p t ( θ ) = p ( θ |D 1: t -1 ) , where the inference becomes

The detailed process can be shown in Appendix A.

In contrast to continual learning, CTTA faces a sequence of learning tasks in test time without any label information, requiring the model to adapt to each novel domain sequentially. In this case, we assume that each domain is i.i.d. and the classes are separable following many unsupervised studies [36, 48, 5], more details about the assumption can be seen in Appendix B.1. We use U = { x n } N n =1 to represent the unlabeled test dataset. The CTTA model is first trained on a source dataset D 0 , and then adapted to unlabeled test domains starting from U 1 . For the t -th adaptation, we have

Similarly, we can set the last posterior to be the current prior, i.e. , p t ( θ ) = p ( θ |U 1: t -1 ∪ D 0 ) and p 1 ( θ ) = p ( θ |D 0 ) . However, employing BI for adaptation on unlabeled testing data can result in untrustworthy posterior estimates. Therefore, during subsequent adaptation, the untrustworthy posterior automatically transform into unreliable priors, leading to error accumulation. In other words,

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

Source Prior

EMA

Student Posterior

Next

Teacher Prior

Test Data

Mix

Teacher Prior

an unreliable prior p t ( θ ) will make the current posterior even less trustworthy. Moreover, the joint likelihood p ( U t | θ ) for t > 0 is intractable on unlabeled data.

To make the BI feasible in CTTA task, in this paper, we transform the question to an easy-to-compute form. Referring to [20], the unsupervised inference can be transformed into

where H denotes the conditional entropy and λ is a scalar hyperparameter to weigh the entropy term. This simple form reveals that the prior belief about the conditional entropy of labels is given by the inputs. The observation of the input U provides information on the drift of the input distribution, which can be used to update the belief over the learned parameters θ through Eq. (4). Consequently, this allows the utilization of unlabeled data for BI. More detailed derivations can be seen in Appendix B.2.

In a BNN, the posterior distribution is often intractable and some approximation methods are required, even when calculating the initial posterior. In this paper, we leverage online VI, as it typically outperforms the other methods for complex models in the static setting [4]. VI defines a variational distribution q ( θ ) to approxmiate the posterior p ( θ |U ) . The approximation process is as follows.

where Q is the distribution searching space and Z t is the intractable normalizing hyperparameter. Thus, referring to the derivations in Appendix C, the ELBO is computed by

Optimizing with Eq. (6) makes model adapt to domain shift. While VI offers a good framework for measuring uncertainty in CTTA, it is noteworthy that VI does not directly address the issue of unreliable priors. The error accumulation remains a significant concern.

Despite this, the form of the ELBO in variational inference offers a pathway for mitigating the impact of unreliable priors. In Eq. (6), the entropy term may result in overly confident predictions that are less calibrated, while the KL term may be directly affected by an unreliable prior. In the following section, we will discuss how to solve the problems when computing the two terms.

4 Adaptation and Inference in VCoTTA

4.1 Entropy term: VI by Mean-Teacher Architecture

In the above section, we introduce the VI in CTTA but challenges remain, i.e. , the unreliable prior.

To mitigate the challenge in the entropy term, we adopt a Mean-Teacher (MT) structure [47] in the

Bayesian inference process. MT is initially proposed in semi-supervised and unsupervised learning, where the teacher model guides the unlabeled data, helping the model generalize and improve

performance with the utilization of large-scale unlabeled data.

MT structure is composed of a student model and a teacher model, where the student model learns from the teacher and the teacher updates using Exponential Moving Average (EMA) [24]. In VI, the student is set to be the variational distribution q ( θ ) , which is a Gaussian mean-field approximation for its simplicity. It is achieved by stacking the biases and weights of the network as follows.

where d denotes each dimension of the parameter. The teacher model ¯ p ( θ ) (we use bar to distinguish 157 the general prior) is also a Gaussian distribution. Thus, the student model is updated by aligning it 158 with the teacher model through the use of a cross-entropy (CE) loss 159

In our implementation, we also try to use Symmetric Cross-Entropy (SCE) [53] in CTTA,

SCE balances the gradient for high and low confidence, benefiting the unsupervised learning.

4.2 KL term: Mixture-of-Gaussian Prior

For the KL term, to reduce the impact of unreliable prior, we propose a mixing-up approach to combining the teacher and source prior adaptatively. The source prior is warmed up upon the pretrained deterministic model p 1 ( θ ) = p ( θ |D 0 ) (see Sec. 4.3.1). The teacher model ¯ p t ( θ ) is updated by EMA (see Sec. 4.3.3). We assume that the prior should be the mixture of the two Gaussian priors. Using only the source prior, the adaptation is limited. While using only the teacher prior, the prior is prone to be unreliable.

We use the mean entropy derived from a given serious data augmentation to represent the confidence of the two prior models, and mix up the two priors with a modulating factor

where I denotes augmentation types. θ 0 and ¯ θ are the parameters of the source model and the teacher model. τ means the temperature factor. Thus, as shown in Fig. 3(b), the current prior p t ( θ ) is set to the mixture of priors as

In the VI, we use the upper bound to update the KL term [31] (see Appendix D.1) for simplicity,

Furthermore, we also improve the teacher-student alignment in the entropy term (see Eq. (9)) by picking up the augmented logits with a larger confidence than the raw data. That is, we replace the teacher log-likelihood log ¯ p ( x | θ ) by

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

where, for brevity, we let ¯ p ( x ′ i ) = ¯ p ( x ′ i | θ ) and ¯ p ( x ) = ¯ p ( x ) | θ ) in short. f ( · ) is the confidence 178 function. ϵ denotes the confidence margin and 1 ( · ) is an indicator function. Eq. (13) can be regarded 179 as a filter, meaning that for each sample, the reliable teacher is represented by the average of its 180 augmentations with ϵ more confidence. In Appendix D.2, we prove that the proposed mixture-of181 Gaussian is benifical to CTTA. In Appendix E.1, we discuss the influence of different ϵ . 182

183

184

185

186

187

4.3 Adaptation and Inference

4.3.1 Variational Warm-up

To obtain a source BNN, instead of training a model from scratch on the source data D 0 , we transform a pretrained deterministic CNN to a BNN by variational warm-up strategy. Specifically, we leverage the local reparameterization trick [27] to add stochastic parameters, and warm up the model:

where p ( θ ) represents the prior distribution, say the pretrained deterministic model. Eq. (14) denotes 188 a standard VI on the source data, and we optimize the ELBO to obtain the variational distribution [49]. 189

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

The warm-up strategy is a common approach in TTA and CTTA tasks to further build knowledge structure for the source model, such as [26, 45, 11, 8]. Some other methods may not use warm-up but still use the source data, such as [39]. The warm-up strategy

Algorithm 1 Variational warm-up

1: Input: Source data D 0 , pretrained model p 0 ( θ )

2: Initialize prior distribution p ( θ ) with p 0 ( θ )

3: Update p ( θ |D 0 ) ≈ q 0 ( θ ) by p ( θ ) and D 0 using Eq. (14)

4: Output: Source prior p 1 ( θ ) = p ( θ |D 0 )

uses the source data only before deploying the model to CTTA scenario, and it is regarded as a part of pretraining. All of these methods using source data are operationalized in source-free at test time and find it is beneficial to CTTA. We use the warm-up to inject the uncertainties into a given source model, i.e., turning an off-the-shelf pretrained CNN model into a pretrained BNN model. This is convenient to obtain a pretrained BNN, because the warm-up strategy uses only a few epochs. We offer more discussions and experiments on the proposed variational warm-up strategy in Appendix F.

4.3.2 Student update via VI

The student model q t ( θ ) is adapted by approximating using Eq. (5), and is optimized on:

where ¯ p ′ t is the current augmented teacher model in Eq. (13), and p 1 ( θ ) ≈ q 0 ( θ ) , ¯ p t ( θ ) ≈ ¯ q t ( θ ) . The KL term between two Gaussians can be computed in a closed form.

4.3.3 Teacher update via EMA

The teacher model is updated using EMA. Let ( µ , σ ) and (¯ µ , ¯ σ ) be the mean and standard deviation of the student and teacher model, respectively. At test time, the teacher model ¯ q t ( θ ) is updated by

Although the std is not used in the cross entropy to compute the likelihood, the teacher prior distribution is important to adjust the student distribution via the KL term.

4.3.4 Model inference

At any time, CTTA model needs to predict and adapt to the unlabeled test data. In our VCoTTA, we also use the mixed prior to serve as the inference model. That is, for a test data point x , the model inference is represented by

For the data prediction, the model only uses the expectation to reduce the stochastic, but leverages stochastic dynamics in domain adaptation.

4.3.5 The algorithm

We illustrate the whole algorithm in Algorithm 2. We first transform an off-theshelf pretrained model into BNN via the variational warm-up strategy (Sec. 4.3.1). After that, we obtain a BNN, and for each domain shift, we forward and adapt each test data point in an MT architecture. For a data point x , we first predict the class label using the mixture of the source model and the teacher model (Sec. 4.3.4). Then, we update the student model using VI, where we use cross entropy to compute the entropy term and use the mixture of

Algorithm 2 Variational CTTA

1: Input: Source data D 0 , pretrained model p 0 ( θ ) , Unlabeled test data from different domain U 1: T

2: p 1 ( θ ) = Variational warm-up ( D 0 , p 0 ( θ )) . // Alg. 1

3: for Domain shift t = 1 to T do

4: for Test data x ∼ U t do

5: Model predict for x (Eq. (17))

6: Update student model using x (Eq. (15))

7: Update teacher model via EMA (Eq. (16))

8: end for

9: end for

priors for the KL term (Sec. 4.3.2). Finally, we update the BNN teacher model via EMA (Sec. 4.3.3). See more details in Appendix G. The process is feasible for any test data without labels.

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

5 Experiment

5.1 Experimental Setting

Dataset . In our experiments, we employ the CIFAR10C, CIFAR100C, and ImageNetC datasets as benchmarks to assess the robustness of classification models. Each dataset comprises 15 distinct types of corruption, each applied at five different levels of severity (from 1 to 5). These corruptions are systematically applied to test images from the original CIFAR10 and CIFAR100 datasets, as well as validation images from the original ImageNet dataset. For simplicity in tables, we use C1 to C15 to represent the 15 types of corruption, i.e. , C1: Gaussian, C2: Shot, C3: Impulse C4: Defocus, C5: Glass, C6: Motion, C7: Zoom, C8: Snow, C9: Frost, C10: Fog, C11: Brightness, C12: Contrast, C13: Elastic, C14: Pixelate, C15: Jpeg.

Pretrained Model . Following previous studies [50, 51], we adopt pretrained WideResNet-28 [57] model for CIFAR10to-CIFAR10C, pretrained ResNeXt-29 [54] for CIFAR100-to-CIFAR100C, and standard pretrained ResNet-50 [21] for ImageNet-to-ImagenetC. Note in our VCoTTA [51], we further warm up the pretrained model to obtain the stochastic dynamics for each dataset. Similar to CoTTA, we update all the trainable parameters in all experiments. The augmentation number is set to 32 for all compared methods that use the augmentation strategy.

5.2 Methods to be Compared

We compare our VCoTTA with multiple state-of-the-art (SOTA) methods. SOURCE denotes the 253 baseline pretrained model without any adaptation. BN [30, 43] keeps the network parameters frozen, 254 but only updates Batch Normalization. TENT [50] updates via Shannon entropy for unlabeled 255 test data. CoTTA [51] builds the MT structure and uses randomly restoring parameters to the 256 source model. SATA [6] modifies the batch-norm affine parameters using source anchoring-based 257 self-distillation to ensure the model incorporates knowledge of newly encountered domains while 258 avoiding catastrophic forgetting. SWA [55] refines the pseudo-label learning process from the 259 perspective of the instantaneous and long-term impact of noisy pseudo-labels. PETAL [2] tries to 260 estimate the uncertainty in CTTA, which is similar to BNN, but it ignores the unreliable prior problem. 261 All compared methods adopt the same backbone, pretrained model and hyperparameters. 262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

5.3 Comparison Results

We show the major comparisons with the SOTA methods in Tables 1, 2 and 3. We have the following observations. First, no adaptation at the test time (SOURCE) suffers from serious domain shift, which shows the necessity of the CTTA. Second, traditional TTA methods that ignore the continual shift in test time perform poorly such as TENT and BN. We also find that simple Shannon entropy is effective in the first several domain shifts, especially in complex 1,000-classes ImageNetC, but shows significant performance drops in the following shifts. Third, the mean-teacher structure is very useful in CTTA, such as COTTA and PETAL, which means that the pseudo-label is useful in domain shift. In the previous method, the error accumulation leads to the unreliable pseudo labels, then the model may get more negative transfers in CTTA along the timeline. The proposed VCOTTA outperforms other methods on all the three datasets, such as 13.1% vs. 15.3% (SWA) on CIFAR10C, 28.4% vs. 30.0% (SWA) on CIFAR100C and 64.2% vs. 66.7% (COTTA) on ImageNetC. We hold the opinion that the prior will inevitably drift in CTTA, but VCOTTA slows down the process via the prior mixture. We also find that the superiority is more obvious in the early adaptation, which may be influenced by the different corruption orders. We analyze the order problem in Appendix H.

5.4 Ablation Study

We evaluate the two components in Table 4, i.e. , the Variational Warm-Up (VWU) and the Symmetric Cross-Entropy (SCE) via ablation. The ablation results show that the two components are both important for VCOTTA. First, the VWU is used to inject stochastic dynamics into an off-the-shelf pretrained model. Without the VWU, the performance of VCOTTA drops to 18.4% from 13.9% on CIFAR10C, 31.5% from 28.8% on CIFAR100C and 68.1% from 64.2% on ImageNetC. Also, the SCE can further improve the performance on CIFAR10C and CIFAR100C, because SCE balances the gradient for high and low confidence predictions. We also find that SCE is ineffective for complex ImageNetC, and the reason may be the class sensitivity imbalance, causing the model to lean more towards one direction during optimization.

5.5 Mixture of Priors

In Sec. 4.2, we introduce a Gaussian mixture strategy, where the current prior is approximated as the weighted sum of the source prior and the teacher prior. The weights are determined by computing the entropy over multiple augmentations of two models. To assess the effectiveness of these weights, we compare them with three naive weighting configurations: using only the source model, using only the teacher model, and a simple average with equal weights for both models. The results, as presented in Table 5, reveal that relying solely on the source model or the teacher model (i.e., weighting with (1 , 0) and (0 , 1) ) results in suboptimal performance. Additionally, naive weighting with equal contributions from both models (i.e., (0 . 5 , 0 . 5) ) proves ineffective for CTTA due to the inherent uncertainty in both models. In contrast, the proposed adaptive weights for the Gaussian mixture in CTTA demonstrate its effectiveness. This underscores the significance of striking a balance between the two prior models in

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

an unsupervised environment. The trade-off implies the need to discern when the source model's knowledge is more applicable and when the teacher model's shifting knowledge takes precedence.

5.6 Uncertainty Estimation

To evaluate the uncertainty estimation, we use negative loglikelihood (NLL) and Brier Score (BS) [3]. Both NLL and BS are proper scoring rules [19], and they are minimized if and only if the predicted distribution becomes identical to the actual distribution:

where D test denotes the test set, i.e. , the unsupervised test dataset U with labels. We evaluate NLL and BS with a severity level of 5 for all corruption types, and the compared results with SOTAs are shown in Table 6. We have the following observations. First, most methods suffer from low confidence in terms of NLL and BS because of the drift priors, where the model is unreliable gradually, and the error accumulation makes the model perform poorly. Our approach outperforms most other approaches in terms of NLL and BS, demonstrating the superiority in improving uncertainty estimation. We also find that PETAL [2] shows good NLL and BS, because PETAL forces the prediction over-confident to unreliable priors, thus PETAL shows unsatisfactory results on adaptation accuracy, such as 31.5% vs. 28.4% (Ours) on CIFAR100C.

5.7 Gradually Corruption

We also show gradual corruption results instead of constant severity in the major comparison, and the results are reported in Table 7. Specifically, each corruption adopts the gradual changing sequence: 1 → 2 → 3 → 4 → 5 → 4 → 3 → 2 → 1 , where the severity level is the lowest 1 when corruption type changes, therefore, the type change is gradual. The distribution shift within each type is also gradual. Under this situation, our VCoTTA also outperforms other methods, such as 8.9% vs. 10.5% (PETAL) on CIFAR10C, and 24.4% vs. 26.3% (COTTA) on CIFAR100C. The results show that the proposed VCOTTA based on BNN is also effective when the distribution change is uncertain.

6 Conclusion and Limitation

Conclusion : In this paper, we proposed a variational Bayesian inference approach, termed VCoTTA, to estimate uncertainties in CTTA. At the pretrained stage, we first transformed an off-the-shelf pretrained deterministic CNN into a BNN using a variational warm-up strategy, thereby injecting uncertainty into the source model. At the test time, we implemented a mean-teacher update strategy, where the student model is updated via variational inference, while the teacher model is refined by the exponential moving average. Specifically, to update the student model, we proposed a novel approach that utilizes a mixture of priors from both the source and teacher models. Consequently, the ELBO can be formulated as the cross-entropy between the student and teacher models, combined with the KL divergence of the prior mixture. We demonstrated the effectiveness of the proposed method on three datasets, and the results show that the proposed method can mitigate the issue of unreliable prior within the CTTA framework.

Limitation : The efficacy of the proposed method relies on injecting uncertainty into the model during the pre-training phase, which may be unavailable in scenarios where pretraining is already completed, and original data is inaccessible. Additionally, constructing and training BNN models are inherently more complex compared to CNNs, highlighting the importance of enhancing computational efficiency. The Gaussian mixture method relies on multiple data augmentations, which also incurs computational costs. Future endeavors could explore more efficient approaches for Gaussian mixture.

340

341

342

343

344

345

346

347

348

References

George EP Box and George C Tiao. Bayesian inference in statistical analysis . John Wiley & Sons, 2011.

Dhanajit Brahma and Piyush Rai. A probabilistic framework for lifelong test-time adaptation. In Proceedings of the Computer Vision and Pattern Recognition , 2023.

Glenn W Brier. Verification of forecasts expressed in terms of probability. Journal of the Monthly Weather Review , 78(1):1-3, 1950.

Thang Bui, Daniel Hernández-Lobato, Jose Hernandez-Lobato, Yingzhen Li, and Richard Turner. Deep gaussian processes for regression using approximate expectation propagation. In Proceedings of the International Conference on Machine Learning , 2016.

Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. Deep clustering for unsupervised 349 learning of visual features. In Proceedings of the European Conference on Computer Vision , pages 350 132-149, 2018. 351

Goirik Chakrabarty, Manogna Sreenivas, and Soma Biswas. Sata: Source anchoring and target alignment 352 network for continual test time adaptation. arXiv preprint arXiv:2304.10113 , 2023. 353

Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In 354 Proceedings of the Computer Vision and Pattern Recognition , 2022. 355

Ziyang Chen, Yiwen Ye, Mengkang Lu, Yongsheng Pan, and Yong Xia. Each test image deserves a 356 specific prompt: Continual test-time adaptation for 2d medical image segmentation. arXiv preprint 357 arXiv:2311.18363 , 2023. 358

Sungha Choi, Seunghan Yang, Seokeon Choi, and Sungrack Yun. Improving test-time adaptation via shift359 agnostic weight regularization and nearest source prototypes. In Procedings of the European Conference 360 on Computer Vision , 2022. 361

Thomas M Cover. Elements of information theory . John Wiley & Sons, 1999. 362

Mario Döbler, Robert A Marsden, and Bin Yang. Robust mean teacher for continual and gradual test-time 363 adaptation. In Proceedings of the Computer Vision and Pattern Recognition , 2023. 364

Sayna Ebrahimi, Mohamed Elhoseiny, Trevor Darrell, and Marcus Rohrbach. Uncertainty-guided continual 365 learning with bayesian neural networks. In Procedings of the International Conference on Learning 366 Representations , 2019. 367

Sebastian Farquhar and Yarin Gal. A unifying bayesian view of continual learning. arXiv preprint 368 arXiv:1902.06494 , 2019. 369

Vincent Fortuin, Adrià Garriga-Alonso, Sebastian W Ober, Florian Wenzel, Gunnar Ratsch, Richard E 370 Turner, Mark van der Wilk, and Laurence Aitchison. Bayesian neural network priors revisited. In 371 Procedings of the International Conference on Learning Representations , 2021. 372

Karl Friston, Jérémie Mattout, Nelson Trujillo-Barreto, John Ashburner, and Will Penny. Variational free 373 energy and the laplace approximation. Journal of the Neuroimage , 34(1):220-234, 2007. 374

Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty 375 in deep learning. In Procedings of the International Conference on Machine Learning , 2016. 376

Yossi Gandelsman, Yu Sun, Xinlei Chen, and Alexei Efros. Test-time training with masked autoencoders. 377 In Procedings of the Advances in Neural Information Processing Systems , 2022. 378

Andrew Gelman, John B Carlin, Hal S Stern, and Donald B Rubin. Bayesian data analysis . Chapman and 379 Hall/CRC, 1995. 380

Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal 381 of the American Statistical Association , 102(477):359-378, 2007. 382

Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. In Proceedings 383 of the Advances in Neural Information Processing Systems , 2004. 384

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 385 In Proceedings of the Computer Vision and Pattern Recognition , 2016. 386

José Miguel Hernández-Lobato and Ryan Adams. Probabilistic backpropagation for scalable learning of 387 bayesian neural networks. In Procedings of the International Conference on Machine Learning , 2015. 388

389

390

391

Hengguan Huang, Xiangming Gu, Hao Wang, Chang Xiao, Hongfu Liu, and Ye Wang. Extrapolative continuous-time bayesian neural network for fast training-free test-time adaptation. In Proceedings of the Advances in Neural Information Processing Systems , 2022.

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

J Stuart Hunter. The exponentially weighted moving average. Journal of the Quality Technology , 18(4):203210, 1986.

Vidit Jain and Erik Learned-Miller. Online domain adaptation of a pre-trained cascade of classifiers. In Proceedings of the Computer Vision and Pattern Recognition , 2011.

Sanghun Jung, Jungsoo Lee, Nanhee Kim, Amirreza Shaban, Byron Boots, and Jaegul Choo. Cafa: Class-aware feature alignment for test-time adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 19060-19071, 2023.

Durk P Kingma, Tim Salimans, and Max Welling. Variational dropout and the local reparameterization trick. In Proceedings of the Advances in Neural Information Processing Systems , 2015.

Richard Kurle, Botond Cseke, Alexej Klushyn, Patrick Van Der Smagt, and Stephan Günnemann. Continual learning with bayesian neural networks for non-stationary data. In Procedings of the International Conference on Learning Representations , 2019.

Dong-Hyun Lee. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on Challenges in Representation Learning, International Conference on Machine Learning , volume 3, page 896, 2013.

Zhizhong Li and Derek Hoiem. Learning without forgetting. Journal of the IEEE Transactions on Pattern Analysis and Machine Intelligence , 40(12):2935-2947, 2017.

GuoJun Liu, Yang Liu, MaoZu Guo, Peng Li, and MingYu Li. Variational inference with gaussian mixture model and householder flow. Journal of the Neural Networks , 109:43-55, 2019.

Yuejiang Liu, Parth Kothari, Bastien Van Delft, Baptiste Bellot-Gurlet, Taylor Mordan, and Alexandre Alahi. Ttt++: When does self-supervised test-time training fail or thrive? In Procedings of the Advances in Neural Information Processing Systems , 2021.

Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. Unsupervised domain adaptation with residual transfer networks. In Proceedings of the Advances in Neural Information Processing Systems , 2016.

Christos Louizos and Max Welling. Multiplicative normalizing flows for variational bayesian neural networks. In Procedings of the International Conference on Machine Learning , 2017.

Wesley J Maddox, Pavel Izmailov, Timur Garipov, Dmitry P Vetrov, and Andrew Gordon Wilson. A simple baseline for bayesian uncertainty in deep learning. In Proceedings of the Advances in Neural Information Processing Systems , 2019.

David J Miller and Hasan Uyar. A mixture of experts classifier with learning based on both labelled and unlabelled data. In Proceedings of the Advances in Neural Information Processing Systems , 1996.

Chaithanya Kumar Mummadi, Robin Hutmacher, Kilian Rambach, Evgeny Levinkov, Thomas Brox, and Jan Hendrik Metzen. Test-time adaptation to distribution shift by confidence maximization and input transformation. arXiv preprint arXiv:2106.14999 , 2021.

Cuong V Nguyen, Yingzhen Li, Thang D Bui, and Richard E Turner. Variational continual learning. In Proceedings of the International Conference on Learning Representations , 2018.

Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In Proceedings of the International Conference on Machine Learning , pages 16888-16905, 2022.

Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. In Proceedings of the Advances in Neural Information Processing Systems , 2019.

Hippolyt Ritter, Aleksandar Botev, and David Barber. A scalable laplace approximation for neural networks. 436 In Procedings of the International Conference on Learning Representations , 2018. 437

438

439

Masa-Aki Sato. Online model selection based on the variational bayes. Journal of the Neural Computation , 13:1649-1681, 2001.

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

484

Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. In Proceedings of the Advances in Neural Information Processing Systems , 2020.

Yoram Singer and Manfred KK Warmuth. Batch and on-line parameter estimation of gaussian mixtures based on the joint entropy. In Procedings of the Advances in Neural Information Processing Systems , 1998.

Junha Song, Jungsoo Lee, In So Kweon, and Sungha Choi. Ecotta: Memory-efficient continual test-time adaptation via self-distilled regularization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 11920-11929, 2023.

Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. In Procedings of the International Conference on Machine Learning , 2020.

Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Proceedings of the Advances in Neural Information Processing Systems , 2017.

Jesper E Van Engelen and Holger H Hoos. A survey on semi-supervised learning. Machine Learning , 109(2):373-440, 2020.

Chong Wang, John Paisley, and David M Blei. Online variational inference for the hierarchical dirichlet process. In Proceedings of the International Conference on Artificial Intelligence and Statistics , 2011.

Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully testtime adaptation by entropy minimization. In Proceedings of the International Conference on Learning Representations , 2020.

Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proceedings of the Computer Vision and Pattern Recognition , 2022.

Yanshuo Wang, Jie Hong, Ali Cheraghian, Shafin Rahman, David Ahmedt-Aristizabal, Lars Petersson, and Mehrtash Harandi. Continual test-time domain adaptation via dynamic sample selection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 1701-1710, 2024.

Yisen Wang, Xingjun Ma, Zaiyi Chen, Yuan Luo, Jinfeng Yi, and James Bailey. Symmetric cross entropy for robust learning with noisy labels. In Proceedings of the Computer Vision and Pattern Recognition , 2019.

Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. In Proceedings of the Computer Vision and Pattern Recognition , 2017.

Xu Yang, Yanan Gu, Kun Wei, and Cheng Deng. Exploring safety supervision for continual test-time domain adaptation. In Proceedings of the International Joint Conference on Artificial Intelligence , 2023.

Longhui Yuan, Binhui Xie, and Shuang Li. Robust test-time adaptation in dynamic scenarios. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1592215932, 2023.

Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Procedings of the British Machine Vision Conference , 2016.

Marvin Zhang, Sergey Levine, and Chelsea Finn. Memo: Test time robustness via adaptation and augmentation. In Procedings of the Advances in Neural Information Processing Systems , 2022.

Tingting Zhao, Zifeng Wang, Aria Masoomi, and Jennifer Dy. Deep bayesian unsupervised lifelong learning. Journal of the Neural Networks , 149:95-106, 2022.

Aurick Zhou and Sergey Levine. Bayesian adaptation for covariate shift. In Proceedings of the Advances in Neural Information Processing Systems , 2021.

485

486

487

488

489

490

491

492

493

Variational Continual Test-Time Adaptation (Appendix)

A Bayesian Inference (BI) in Traditional CL and CTTA

As described in Sec. 3, we first illustrate the BI has been studied in traditional Continual Learning (CL) methods. In this section, we compare the BI in CL and CTTA in detail and show the differences with some related works. The comparison can be seen in Fig. 3. For the CL, BI is conducted by the posterior propagation, that is, the prior of next task is equal to the current posterior. This is feasible in supervised CL, where the data label is provided. For the CTTA, the posterior is not trustworthy using only pseudo labels to adapt to a new domain. Thus, propagate the untrustworthy posterior to the next stage would make unreliable prior, which will result in error accumulation. In the proposed VCoTTA, we propose to solve the problem via enhancing the two terms in VI (see Sec. 4).

Prior

Equal

Posterior

Next Prior

Labeled

Data

(a) BI in continual learning

Source Prior

EMA

Posterior

Next

Teacher Prior

Unlabeled

Data

Teacher Prior

Mix

(b) BI in CTTA

VCL [38] is a classic CL study that uses VI, our work is also inspired by VCL but has the following 494 difference. (1) The tasks are different : VCL studies supervised CL task, while our VCoTTA studies 495 unsupervised CTTA task. (2) The challenges are differnt : CL only suffers from catastrophic forgetting 496 (CF), while CTTA sufffers from both CF and error accumulation. (3) Ways of BI are different : To 497 conduct BI, one needs to compute prior and likelihood. For the prior, the current prior of VCL is set 498 to be the previous posterior, while in CTTA such a prior may be unreliable. For the likelihood, VCL 499 can directly compute likelihood, CTTA is under unsupervised setting, thus in our work, we deduce 500 the BI in CTTA using conditional entropy. (4) The update strategies are different : To reduce error 501 accumulation in unsupervised scenario, we employ a mean-teacher update strategy using VI for the 502 student model and exponential moving average for the teacher model, and compute a prior mixture 503 to guide the student update. Moreover, VCL maintains an extra coreset from the training set, while 504 VCoTTA never store any data during the test time. 505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

520

521

522

523

524

525

526

527

528

529

530

531

532

533

534

535

536

537

538

539

540

541

542

543

544

545

546

547

We also find another recent work named PETAL [2] that estimates uncertainties in CTTA. The BI formulation is similar between PETAL and ours, which is derived from [20], but PETAL use different method to conduct the inference: (1) PETAL only uses CNN and does not estimate the model uncertainties, while VCoTTA uses BNN to model the uncertainties during test time. (2) PETAL ignores the unreliable prior in CTTA, and follow the VCL setting that use the previous posterior as the current prior. (3) We conduct BI using variational inference while PETAL use SWAG [35]. SWAG has advantages in terms of computational efficiency and stability during training, especially in scenarios where computational resources are limited. However, SWAG might not handle unreliable priors as effectively as VI since it doesn't explicitly model the posterior distribution. (4) We have compared with PETAL in our experiment (see Tables 1, 2, 3), and our method outperforms PETAL on all datasets.

B CTTA Approximation by BI

B.1 Assumption on Class Separability

In our method, we use the conditional entropy to alternate the intractable computing of likelihood. Note that the use of entropy in unsupervised scenario needs to satisfy the class-separable assumption. In fact, unlabeled data do not convey category information but still carry information. Miller and Uyar [36] theoretically proved that utilizing unlabeled samples to train classifiers can improve classification performance if there is a connection between the target and sample distributions. It is a common practice in unsupervised/semi-supervised learning to establish the relationship between unlabeled data and the target by making some reasonable assumptions to obtain categoryrelevant information from unlabeled data. Common assumptions include the Smoothness assumption , Cluster assumption , Manifold assumption , Low-density separation assumption , etc. For example, the well-known clustering-based methods utilize the cluster assumption to generate pseudo-labels for unsupervised learning [48]. Caron et al. [5] assumes that "the model trained on labeled data will produce high uncertainty estimation for unseen data" in domain adaptation tasks to benefit the classifier from unlabeled data lacking category information.

Bengio et al. in [20] proposed the conditional entropy and point out that "These studies conclude that the (asymptotic) information content of unlabeled examples decreases as classes overlap. Thus, the assumption that classes are well separated is sensible if we expect to take advantage of unlabeled examples." This assumption has been applied to many studies, for example in [29, 33, 60, 2]. In the CTTA task of this paper, as the task progresses, the domain shifts, but the categories in the task remain unchanged. Therefore, under the assumption that unlabeled data contains information, we can reasonably continue to use conditional entropy in the current scenario. To sum up, whether in unsupervised TTA or in the Bayesian field, this assumption is not difficult to achieve or has never been applied. We can quite naturally continue to use this assumption in the context of this paper.

B.2 BI during Test Time

The goal of CTTA is to learn a posterior distribution p ( θ |U 1: T ∪ D 0 ) from a source dataset D 0 , and a sequence of unlabeled test data from U 1 to U T . Following [60], assuming we have multiple input-generating distributions that the source dataset D 0 is drawn from a distribution ϕ , and ˜ ϕ t specifies the shifted of the t -th unlabeled test dataset which we aim to adapt to. Let the parameters of the model be θ ,then following the semi-supervised learning framework [20], we incorporate all input-generating distributions into the belief over the model parameters θ as follows

where the inputs X are sampled i.i.d. from a generative model with parameters ϕ , while the corre548 sponding labels Y are sampled from a conditional distribution p ( Y | X, θ ) , which is parameterized 549 by the model parameters θ . p ( θ ) is a prior distribution over θ . { λ 0 , λ 1 , · · · , λ T } are the factors for 550 approximation weighting. Generally, the entropy term H θ ,ϕ ( Y | X ) represents the cross entropy of 551 the supervised learning, and the entropy term H θ , ˜ ϕ t ( Y | X ) for t > 0 denotes the Shannon entropy of 552 the unsupervised learning. 553

Following [60], we can empirically use a point estimation to get a plug-in Bayesian approach to 554 approximate the above formula: 555

To make the formula feasible to CTTA, that is, no source data is available at the test time, we set 556 λ 0 = 0 . And the source knowledge can be represented by p ( θ |D 0 ) ∝ p ( θ ) ∏ ∀ x,y ∈D 0 p ( y | x, θ ) . 557 Thus, for the t -th test domain, the Bayesian inference in CTTA can be represented as follows: 558

559

560

561

562

563

564

where H ( U t | θ ) = 1 |U t | ∑ ∀ x ∈U t H ( Y | x, θ ) and the above formula can be rewritten in simplicity as

which specifies the Bayesian inference process on continuously arriving unlabeled data in CTTA.

C ELBO of the VI in CTTA

We built VI for CTTA in Sec. 3, where we initialize a variational distribution q ( θ ) to approximate the real posterior. For the test domain t , we optimize the variational distribution as follows:

where Q is the distribution searching space, and p t ( θ ) is the current prior.

Following the definition of KL divergence and the standard derivation of the Evidence Lower BOund 565 (ELBO) is as the following formulas. Specifically, the KL divergence is expanded as 566

where the first constant term can be reduced in the optimization. Thus, we can optimize the variational 567 distribution via the ELBO: 568

569

570

571

572

573

574

575

576

577

578

579

580

In our case, the former entropy term can be more effectively replaced by the cross entropy or symmetric cross entropy (SCE) between the student model and the teacher model in a mean-teacher architecture (see Sec. 4.1). For the latter KL term, we can substitute a variational approximation that we deem closest to the current-stage prior p t ( θ ) into the KL divergence. When the prior is a multivariate Gaussian distribution, this term can be computed in closed form as

where Σ = diag ( σ 2 ) , k represents the dimensionality of the distributions, tr ( · ) denotes the trace of a matrix, and det( · ) stands for the determinant of a matrix. For the case that the prior is a mixture of Gaussian distributions, we can refer to the next section to get its upper bound.

D Mixture-of-Gaussian Prior

D.1 Upper Bound of the Mixture of Two KL Divergencies

We refer to the lemma that was stated for the mixture of Gaussian in [44]. The KL divergence between two mixture distributions p = ∑ k i =1 α i p i and p ′ = ∑ k i =1 α i p ′ i is upper-bounded by

where α = ( α 1 , α 2 , · · · , α k ) and α ′ = ( α ′ 1 , α ′ 2 , · · · , α ′ k ) are the weights of the mixture components. 581 The equality holds if and only if α i p i / ∑ k j =1 α j p j = α ′ i p ′ i / ∑ k j =1 α ′ j p ′ j for all i . Using the log-sum 582 inequality [10], we have 583

In our algorithm, q ( θ ) is set to be a mixture of Gaussian distributions, i.e. , p t ( θ ) = α · p 1 ( θ ) + (1 -584 α ) · ¯ p t ( θ ) . In the above inequality, let q ( θ ) = ∑ k i =1 α i q ( θ ) , we can get the upper bound of the KL 585 divergence between q ( θ ) and p t ( θ ) : 586

So the lower bound (24) can be redefined as 587

Then, we have obtained a lower bound that can be optimized through closed-form calculations as 588 the source prior distribution q 0 ( θ ) and the teacher prior distribution ¯ q t ( θ ) are multivariate Gaussian 589 distributions, which means we can also optimize L ′ with Eq. (25). 590

591

592

593

594

595

596

597

598

599

600

601

602

603

604

605

606

607

608

609

610

611

612

613

614

615

616

617

618

619

620

621

622

D.2 Advantage of the Mixture of Gaussian Prior

In this subsection, we illustrate why the mixture of Gaussian prior are beneficial to CTTA. First of all, we can start from defining what is a better distribution for CTTA. Assume there exists an ideal prior distribution ˆ p t , which effectively represents the distribution of the model after learning all past knowledge, including that from the source and unlabeled datasets. Then we can use the difference between a distribution and the ideal distribution ˆ p t (here we use KL divergence) to measure the goodness of a distribution, i.e., KL ( ·|| ˆ p t ) .

Generally, neither the source prior p 1 (trained on labeled data) nor the adapted prior ¯ p t (adapt on unlabeled data, being unreliable) can be completely consistent with ˆ p t . Considering that, as t increases, the difference between ¯ p t and ˆ p t will increase without an upper bound due to the error accumulation (since t is infinitely growing). The source prior p 1 cannot adapt to the unlabeled data, but it contains important information from the labeled data, and the ideal distribution cannot forget the source information too much, so we can assume that the difference between p 1 and ˆ p t is a constant, i.e., KL ( p 1 || ˆ p t ) < U , where U is a constant upper bound. Accordingly, it can be considered that mixing the source prior p 1 and the adapted prior ¯ p t in some way is beneficial for reducing KL ( ·|| ˆ p t ) .

In our paper, we consider using a simple Gaussian mixture, i.e., p t = α t p 1 +(1 -α t )¯ p t , where α is computed by Eq. (10). It is easy to illustrate the benefits of this idea using the following inequality:

In Eq. (29), if KL (¯ p t || ˆ p t ) ≥ U , which can be satisfied as mentioned above, then we have

This indicates that the mixed distribution p t is closer to the ideal distribution ˆ p t than the adapted prior ¯ p t . A similar idea can be found in the stochatic restoration in CoTTA [51], where the author randomly restore parts of parameters of the current model into the parameters of source model.

E Augmentation Analysis

In our method, we use the standard augmentation following CoTTA [51]. In this subsection, we analyze the some characteristics via experiments.

E.1 Confidence Margin

First, we analyze the margin ϵ in Eq. (13). We experimentally validate different margins with more choices. Experimental results are shown in Tables 8. The results indicate that different datasets may require different margins to control confidence. Moreover, Eq. (13) signifies that the reliable teacher likelihood is represented by the mean of its augmentations with ϵ more confidence than the teacher itself. Tables 8 illustrates the selection of ϵ in our approach on CIFAR10C, CIFAR100C and ImageNetC. Note that when ϵ = -1 , it means no margin is used and the method will use all augmentated samples, i.e., without using Eq. (13). The results show that the proposed margin can effectively filter out unreliable augmented samples and achieve a better teacher log-likelihood.

623

E.2 Different Number of Augmentation

In our method, we also use augmentation to enhance the confidence. We then evaluate the the number 624 of augmentation in Eq. (10). The results can be seen in Table 9, and shows that increasing the number 625 of augmentations can enhance effectiveness, but this hyperparameter ceases to have a significant 626 impact after reaching 32. 627

F Further Discussion on Variational Warm-up Strategy

We have discussed the Variational Warm-Up (VWU) strategy in Sec. 4.3.1, and explain that the warm-up strategy is a common practice in TTA and CTTA. In this section, we further discuss some attributes of the proposed variational warm-up strategy.

In our method, the VWU strategy is used to turn an off-the-shelf CNN to a pretrained BNN. The advantage of this approach is that pretrained CNNs are readily available (e.g., directly leveraging official models in PyTorch), while pretrained BNNs are challenging to obtain, especially for largescale datasets. Moreover, training BNNs is more difficult compared to training CNNs. Therefore, constructing BNN pretrained models based on existing CNN pretrained models is a feasible approach. Additionally, we find that such a warm-up strategy requires only a few epochs to achieve satisfactory results. To validate the characteristics of the proposed VWU strategy, we designed the following experiments.

F.1 Warm-up on CNN vs. Directly Pretraining BNN

First, we conducted experiments to compare the performance of obtaining pretrained BNN models using the warm-up approach versus directly training the source model with BNN. We pretrain the BNN also use VI as describing in Sec. 4.3.1. The results can be seen in Table 10. As we can see, the results are at the same level, for example VI pretraining is with 13.2% error rate while the proposed VWU achieves 13.1% on CIFAR10C. However, if we direct turn a pretrained CNN to a BNN by adding random stochastic parameters, without warm-up strategy, the results drop to 17.1%. This shows that VWU is a feasible strategy to obtain a pretrained BNN.

F.2 Number of Warm-up Epochs

In our implementation, we employ only a limited number of epochs for variational warm-up, say 5 epochs. This is due to the fact that the pretrained model fits well in CNN, thus requiring minimal adjustments to the mean of BNN. Additionally, the standard deviation (std) is initialized to be small. Consequently, only a small number of iterations are necessary to update the BNN, and the step size is also kept small. Experimentation on the epoch number of variational warm-up reveals that keeping increasing epochs ( > 5 ) will diminishes performance, as shown in Fig. 5.

628

629

630

631

632

633

634

635

636

637

638

639

640

641

642

643

644

645

646

647

648

649

650

651

652

653

654

655

656

657

658

659

660

661

662

663

664

665

666

667

668

669

670

671

672

673

674

675

676

677

678

679

680

681

682

683

684

Error

64.0

64.5

65.0

65.5

66.0

Number of epoch

1

2

3

4

5

6

7

8

9

10

Number of epoch

Error

13.0

13.5

14.0

14.5

15.0

Data portion

1/10

1/4

1/2

1/4

1

15.0 F.3 Only Portion Usage of Source Dataset in Warm-up

Error 13.5 14.0 14.5 1 As we response to the weakness, the warm-up strategy is a common approach in TTA and CTTA tasks and it is regarded as a part of pretraining stage. We also evaluate how if we only use partial data for warm-up, and the results are as follow. The experimental results demonstrate that a moderate reduction in sample size still maintains certain effectiveness of the warmup strategy. However, excessive reduction, such as reducing to 1/10, leads to a certain decline in effectiveness. This is because the warmup strategy aims to incorporate statistical information of the dataset into the model, and insufficient data may result in inaccurate performance.

13.0

1/10

1/4

1/2

1/4

1

Data portion G Recursive Variational Approximation Process in VCoTTA

In this section, we show the algorithmic workflow utilizing variational approximation in VCoTTA.

Before testing time : First, we adopt a variational warm-up strategy to inject stochastic dynamics into the model before adaptation. Given the source dataset D 0 , we can use a variational approximation of p ( θ |D 0 ) as follows

1 where we use the pretrained deterministic model p 0 ( θ ) as the prior distribution.

When the domain shift : Then, at the beginning of the test time, we set the prior in task t as p t ( θ ) = α · p 1 ( θ ) + (1 -α ) · ¯ p t ( θ ) and variational approximation, where p 1 ( θ ) ≈ q 0 ( θ ) and ¯ p t ( θ ) ≈ ¯ q t ( θ ) . For ¯ q t ( θ ) , which means the real-time posterior probability of the teacher model for the t -th test domain, is constantly updated by q t ( θ ) via EMA (see Sec. 4.3.3) during the test phase. Note that we do not have ¯ q t ( θ ) for the first update in the t -th phase. In fact, we use q t -1 ( θ ) construct the prior, thus we have p t ( θ ) ≈ α · p 1 ( θ ) + (1 -α ) · q t -1 ( θ ) . This is the variational distribution that should be used to approximate the prior in the absence of a teacher model in the first step, as well as the approximation that should be used when not employing the MT architecture. Note that the process is not required to inform the model that the domain produces a shift.

During the testing time of a domain : With the approximation to p t ( θ ) and analysis from Appendix B.2, we get q t ( θ ) for student model at the test domain t as follows:

which means, we can recursively derive p t +1 ( θ ) and the following variational distributions, thereby achieving the goal of VCoTTA.

H Different Orders of Corruption

As we discuss in the major comparisons (see Sec 5.3), the performance may be affected by the corruption order. To provide a more comprehensive evaluation of the matter of the order, we conduct

10 different orders from Sec 5.3, and show the average performance of all compared methods. 685 10 independent random orders of corruption are all under the severity level of 5. The results 686 are shown in Table 11. We find that the order of corruption is minor on simple datasets such as 687 CIFAR10C and CIFAR100C, but small std on difficult datasets such as ImageNetC. The proposed 688 VCOTTA outperforms other methods on the average error of CIFAR10C and CIFAR100C under 10 689 different corruption orders, which shows the effectiveness of the prior calibration in CTTA. Moreover, 690 VCOTTA has comparable results with PETAL on ImageNetC, but smaller std over 10 orders, which 691 shows the robustness of the proposed method. 692

I Corruption Loops

In the real-world scenario, the testing domain may reappear in the future. We evaluate the test conditions continually 10 times to evaluate the long-term adaptation performance on CIFAR10C. That is, the test data will be re-inference and re-adapt for 9 more turns under severity 5. Full results can be found in Fig. 6. The results show that most compared methods obtain performance improvement in the first several loops, but suffer from performance drop in the following loops. This means that the model drift can be even useful in early loops, but the drift becomes hard because of the unreliable prior. The results also indicate that our method outperforms others in this long-term adaptation situation and has only small performance drops.

1

2

3

4

5

6

7

8

9

10

13

14

15

16

17

Error

CoTTA

PETAL

VCoTTA

J Experiment on Online Setting

693

694

695

696

697

698

699

700

701

702

CTTA does operate in an online setting, where all testing data is used only once. However, the current 703 focus of CTTA research primarily revolves around batch-mode online settings, with batch sizes 704 typically set to 200 in our experiments like other SOTAs. In CTTA, strict online learning settings 705 where each data point is processed individually are under-researched. In fact, our method can be 706 applied in scenarios with online learning or small batch sizes. However, it's important to note that the 707 batch normalization (BN) layers is disabled when the batch size is 1. We experimented with batch 708 size of 1 on CIFAR10C, and compare the results with some baseline methods. The comparison results 709 are shown in Table 12. The results show that small batch size in CTTA makes worse performance. 710 We believe this is because a small batch size amplifies the uncertainty in model training. 711

712

K Time and Memory Cost

We implement our method using a single RTX-4090 GPU card. We provide the memory and time cost 713 in Table 13. Our proposed VCoTTA method does not offer an advantage in terms of memory usage. 714 This is because in the BNN framework, additional standard deviations are required for implementing 715 local reparameterization tricks. However, during the testing phase, this does not significantly impact 716 the efficiency of the model. This is because during testing, only the student model employs variational 717 inference, which requires uncertainty parameters. 718

719

720

721

722

723

724

725

726

727

728

729

730

731

732

733

734

735

736

737

738

739

740

741

742

743

744

745

746

747

748

749

750

751

752

753

754

755

756

757

758

759

760

761

762

763

764

765

766

NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.

Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:

You should answer [Yes] , [No] , or [NA] .

[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.

Please provide a short (1-2 sentence) justification right after your answer (even for NA).

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] " provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA] " is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.

IMPORTANT, please:

Delete this instruction block, but keep the section heading 'NeurIPS paper checklist" ,

Keep the checklist subsection headings, questions/answers and guidelines below.

Do not modify the questions and only use the provided macros for your answers .

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We made clear claims to illustrate that we evaluate the uncertainty in CTTA task using variational inference.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]

Justification: We discuss the limitation in the last section.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We provide the assumption and proofs mostly in appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We use open-source dataset and provide a anonymous code link.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We provide the anonymous code link.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We follow previous to set the experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We offer the 10 different task orders to reduce the influence of stochastic and provide the avg ± std in Appendix H.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provide the compute resources in Appendix K.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

924

925

926

927

928

929

930

931

932

933

934

935

936

937

938

939

940

941

942

943

944

945

946

947

948

949

950

951

952

953

954

955

956

957

958

959

960

961

962

963

964

965

966

967

968

969

970

971

972

973

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We confirm that we conducted in the paper conform with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: Nor applicable. We study machine learning problem on public dataset such as CIFAR10.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

974

Justification: No such risks.

975

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We referred to open-source code from various methods and developed our own implementation of the core algorithm.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: No new assets will be released.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

1028

Justification: We use public dataset.

1029

Guidelines:

1030

1031

1032

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

1033

1034

1035

1036

1037

1038

1039

1040

1041

1042

1043

1044

1045

1046

1047

1048

1049

1050

1051

1052

1053

1054

1055

1056

1057

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: We do not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a variational continual adaptation method. Where a sequence of test-time domain adaptation problems are shown to a model. More specifically a labeled dataset is given as an initial dataset to learn from, and then afterwards a sequence of unlabelled datasets with domain shifts are presented to the model.\n\nUsing traditional variational continual learning methods will result in error accumulations in the posterior over parameters. So the authors propose a scheme that regularizes against the posterior learned from an initial source dataset. \n\nThe authors propose do away with using sequential Bayesian inference. Instead, the authors use a mix of a source prior (learned using labeled data) and a teacher prior. The teacher prior is an EMA of the previous task’s posterior learned with variational inference. The method the authors propose is named VCoTTA.\n\nSTRENGTHS:\n* The paper is well written and the components of VCoTTA are clearly explained. \n* The authors provide an ablation to demonstrate which design choices worked well, for instance, to demonstrate that the mixture of priors worked well for test-time adaptation.\n\nWEAKNESSES:\nNovelty:\n* In terms of novelty I’m not convinced that there are important applications of continual test-time adaptation in the form of classification tasks derived from CIFAR10 datasets. I could be wrong, but some justification in the paper is required and some more realistic benchmarks would be nice.\n* Maybe I have misunderstood the variational warm-up procedure. But using the MLE estimates to initialize the BNN mean parameters was done in VCL https://arxiv.org/abs/1710.10628 . So this is not a novel idea. Furthermore, there are better ways to initialize a BNN such as using Bayesian linear regression: https://proceedings.mlr.press/v97/rossi19a/rossi19a.pdf.\n\nClarity\n* Why does the teacher model use EMA updates instead of using the inference variational posterior?\n* Why is data augmentation an important component in VCoTTA (Sec 4.2)? This is suddenly presented in the paper without justification.\n\nNotation:\n* In Section 4.2, the title is a “Mixture-of-Gaussian prior”, but Eq 11 is an addition of two priors which are Gaussians by design, so this is a “scale-mixture prior” https://arxiv.org/pdf/1505.05424, not a mixture of Gaussians (https://www.inf.ed.ac.uk/teaching/courses/mlpr/2016/notes/w9b_mixture_models.pdf)?\n* Confusing notation of the source prior: it is denoted as $p_0$ (Fig 2) and $p_1$ (Eq 11), this needs to be consistent.\n\n\nEmpirical weaknesses:\n* No standard errors in the experimental results. So difficult to see which method outperforms another.\n* Uncertainty estimation is not performed with standard methods like ECE or OOD detection like https://arxiv.org/abs/2102.06571. It is unclear to me whether the Brier Score estimates uncertainties.\n\nQUESTIONS:\nPlease see the weaknesses above. Additionally, there are some other minor questions mainly regarding the writing and the questions around implementation:\n* Line 169: what is a “serious data augmentation”?\n* Line 178: what is a confidence function?\n* Eq 13: what is $x’$?\n* Alg 1, line 2: how do you initialize the prior distribution $p(\\theta)$ with the MLE weights from a pre-trained CNN? Do you set the mean variational means to the MLE values and initialize the variances to be some small initial value such as $10^-3$?\n* On the first domain adaptation update of VCOTTA, how is $\\bar{p}_1$ defined since we do not have an initial teacher model yet (Eq 11)?\n\nLIMITATIONS:\nThere is a good discussion on the limitations of VCoTTA.\n\nOne limitation that is not discussed is in the effectiveness of (variational) Bayesian sequential inference methods. Weight space variational inference has been shown to be very difficult to do in practice, https://arxiv.org/abs/2301.01828. So weight space variational inference (without tricks like multi-head networks and coresets) might not be the best choice when wanting to remember a source distribution.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces VCoTTA, a novel variational Bayesian approach to address the Continual Test-Time Adaptation (CTTA) task, which focuses on effective domain adaptation during continuous domain shifts at test time. The authors' main contributions include a method to measure uncertainties in CTTA, addressing the issue of error accumulation due to the use of unlabeled samples. They propose transforming a pretrained deterministic model into a Bayesian Neural Network (BNN) using variational warm-up at the source stage, and employ a mean-teacher update strategy during test time. The approach updates the student model by combining priors from both source and teacher models, with the evidence lower bound formulated as the cross-entropy between student and teacher models, along with the Kullback-Leibler (KL) divergence of the prior mixture. Experimental results on three datasets demonstrate the method's effectiveness in mitigating error accumulation within the CTTA framework.\n\nSTRENGTHS:\nThe paper demonstrates originality through the novel Variational Continual Test-Time Adaptation (VCoTTA) approach, which creatively utilizes Bayesian Inference for Continual Test-Time Adaptation, and employs strategies like the variational warm-up and prior mixture techniques. The quality of the work is evident in its solid theoretical foundation, comprehensive methodology, and empirical validation on multiple datasets. \n\nThe paper's clarity is apparent in its well-structured presentation, use of visual aids, and explicit statement of contributions. The significance of the research is underscored by its practical relevance to risk-sensitive applications, potential for broad applicability, and the reported improvements in predictive accuracy and uncertainty estimation under distribution shifts. By addressing critical challenges in CTTA, such as error accumulation and uncertainty estimation, and bridging Bayesian methods with test-time adaptation, the paper not only advances the current state of the art but also opens up promising avenues for future research. Overall, this work represents a valuable contribution to the field, offering both theoretical insights and practical advancements in continual learning and test-time adaptation.\n\nWEAKNESSES:\nRegarding the computational overhead discussed in the paper, while it is noted that online Variational Inference is employed to make the approach computationally feasible, a detailed analysis of the computational costs associated with VCoTTA is absent. Table 13 presents a comparison of time and memory costs, but the source of these values is unclear. Could you specify which dataset was used for these measurements? Also, is it possible to clarify whether the time and memory comparisons pertain to training or testing phases?\n\nThe manuscript contains several typographical and grammatical errors that need to be addressed. Specifically, brackets are missing in Equation (5) and in the sentence following Equation (13). Could these omissions be corrected to prevent misinterpretation of the mathematical expressions and enhance the clarity of the paper?\n\nThere are multiple grammatical issues that require rectification. The sentence \"MT is initially proposed in semi-supervised and unsupervised learning\" is somewhat unclear. Could this be rephrased for better coherence? Additionally, the sentence \"We use the mean entropy derived from a given *serious* data augmentation to represent the confidence of the two prior models, and mix up the two priors with a modulating factor\" appears to contain a typographical error and could be better structured. Could these issues be addressed to improve the readability and accuracy of the text?\n\nThe heading for Section 5.7 seems to not accurately reflect the content discussed within. Could this heading be revised to more accurately convey the main topics or findings of the section, thereby ensuring clarity and relevance for the reader?\n\nThe explanation of how MT operates in semi-supervised and unsupervised learning settings appears incomplete and potentially misleading. The current statement, \"where the teacher model guides the unlabeled data, helping the model generalize and improve performance with the utilization of large-scale unlabeled data,\" lacks specificity. Could you specify which model (teacher or student) benefits from this guidance and in what manner? Additionally, the phrase \"where the teacher model guides the unlabeled data\" seems incorrect. Could this be clarified or corrected to accurately reflect the operational dynamics of the MT framework?\n\nQUESTIONS:\nPlease provide answers to the questions/suggestions provided in the Weaknesses section.\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a method to continually adapt a pre-trained classifier to an unlabeled stream of test data. They address the problem of continual test-time adaptation through the lens of Bayesian deep learning. Their method consists of three main components: (1) a variational warm-up strategy to turn any source model into a Bayesian Neural Network, (2) a mixing strategy between the source model and the last posterior to leverage the trade-off between adaptation and forgetting, and (3) a modified entropy term that is symmetric and incorporates data augmentations. The authors compare their method on standard CIFAR-C and Imagenet-C datasets to a set of TTA baselines. The paper also includes ablation studies on the components and an evaluation of uncertainty estimates.\n\nSTRENGTHS:\n- The paper tackles one of the most relevant problems in continual domain adaptation, namely the trade-off between agile adaptation while preventing the forgetting of the source model. It does so by constituting a mixture of the source and last adapted model in a VI framework, which is novel to my knowledge. (originality)\n- Further, in a setting where robustness is crucial and therefore uncertainty quantification can be helpful, the combination of Bayesian deep learning and continual test-time adaptation is interesting and insightful. (originality)\n- The methodological backbone is accompanied by insightful ablation studies that highlight the significance of the different parts of the paper’s contribution. (quality)\n- The paper is, in most parts, pleasant to read. The notation is clear and consistent, and the reader is well guided through the different sections. (clarity)\n- The paper presents clear experimental evidence in support of the method. The experiments show an improvement in adaptation accuracy on already quite saturated datasets (up to 1.8% points on CIFAR-10-C). VCoTTA also seems to be advantageous on most corruption types. (significance)\n\nWEAKNESSES:\nThe paper presents strong evidence in support of the proposed method. However, it is left unclear to me why the method performs so much better than previous approaches.\n\n- The method consists of a range of specific components. However, in some cases, the specific design of the components is not clearly motivated. In particular, equations 10 and 13 lack supporting citations or explanations. Why have exactly these formulations been chosen?\n- I’d like to get more clarity on the difference between this paper and the original CoTTA work, as it seems to me there are certain components in common (e.g., student-teacher approach, EMA). More precisely, could you please highlight the difference in the update equations between the two papers? My understanding is that adding the VI framework notably changes (i) the optimization objective by adding the KL term (instead of solely minimizing entropy) and (ii) the predictions by marginalizing out the model parameters. Where else does the VI framework contribute to differences?\n\nQUESTIONS:\n- Recent work has explored the degradation of CTTA methods after a long adaptation period [1]. Given that the paper argues that the mixed prior between the adapted and source model prevents forgetting, it would be interesting to see for how long the adaptation can be successful. Is there any experimental evidence on the robustness over time of the adaptation method? See [1] for an example of a suitable benchmark dataset.\n- What is the motivation behind Equation 13? In other words, why would we only want to have log-likelihood terms of “confident augmentations”? I would have assumed that having a large pool of augmentations and averaging them all would give more robust predictions. An ablation study on the design of this term would also be insightful.\n- Could the method suffer from mode averaging in Equation 11?\n- Am I assuming correctly that only one gradient step is performed per test batch?\n\n[1] Press, Ori, et al. \"Rdumb: A simple approach that questions our progress in continual test-time adaptation.\" Advances in Neural Information Processing Systems 36 (2024).\n\nLIMITATIONS:\nThe authors have listed limitations including computational efficiency and the need for access to the source data at adaptation time", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a variational Bayesian approach to handle uncertainties in continual test-time adaptation (CTTA). The source pretrained model is made Bayesian by variational warm and a mean-teacher update strategy is used at test time. To avoid drift due to uncertainty of priors using only unlabeled data at test time, the paper proposed to update the student model by combining priors from both the source and teacher models. The evidence lower bound is formulated as the cross-entropy between the student and teacher models, along with the Kullback-Leibler (KL) divergence of the prior mixture. Experimental results on three datasets demonstrate the method’s effectiveness in mitigating error accumulation within the CTTA framework.\n\nSTRENGTHS:\n- Novelty: Bayesian approach in Continual Learning is a principled and elegant approach to the problem which this paper is relying on. In CTTA, there are additional issues due to the uncertainty of the prior distributions using only the unlabeled data from unknown domains. This paper presents a novel solution by using adaptive mixture prior models and student-teacher update on top of an existing framework.\n\n- Relevance: CTTA is a topic that can interest a general audience, and the  Bayesian and variational framekwork can also be of interest to many.\n\nWEAKNESSES:\n- While Bayesian approach is nice in principle, it can be computationally demanding and offer little benefit in practice. Most of the existing CTTA methods are computationally and memory efficient, whereas this method present an opposite end of the spectrum. While the reported results are impressive, it is unclear to me why the proposed method is superior to other SOTA methods.\n- The hyperparameter selection process is not addressed in the paper, which is critical in TTA where all hyperparameters should be predetermined before data access. How are they chosen?\n\nQUESTIONS:\n- What is the rough analysis of the space and time complexity of the proposed approach, separate for off-line and online phases? There is only one incomplete empirical comparison in the appendix, which leave me curious how the space and time cost change in different settings (e.g., batch size, training set size, model size, etc).\n\n- In the experiments, can resetting to source simply solve the drift issue and achieve much better results than the naive results in the tables?\n\n- For the mixture rate alpha, what is the best result with a non-adaptive alpha?\n\nLIMITATIONS:\nLimitations are mentioned in the paper, albeit very brief. Overall, the proposed method is more complex and demanding (such as requiring a pretrained probabilistic model or source data) for TTA applications. Perhaps the proposed approach may work even better with UDA or other CL scenarios with (partially available) target labels?", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}