SODA: Robust Training of Test-Time Data Adaptors
Authors: Zige Wang
Yonggang Zhang
Zhen Fang
Long Lan
Wenjing Yang
Bo Han
Venue: NeurIPS 2023 poster (2023)
Keywords: test-time data adaptation
zeroth-order optimization
out-of-distribution generalization

Abstract: Adapting models deployed to test distributions can mitigate the performance degradation caused by distribution shifts. However, privacy concerns may render model parameters inaccessible. One promising approach involves utilizing zeroth-order optimization (ZOO) to train a data adaptor to adapt the test data to fit the deployed models. Nevertheless, the data adaptor trained with ZOO typically brings restricted improvements due to the potential corruption of data features caused by the data adaptor. To address this issue, we revisit ZOO in the context of test-time data adaptation. We find that the issue directly stems from the unreliable estimation of the gradients used to optimize the data adaptor, which is inherently due to the unreliable nature of the pseudo-labels assigned to the test data. Based on this observation, we propose pseudo-label-robust data adaptation (SODA) to improve the performance of data adaptation. Specifically, SODA leverages high-confidence predicted labels as reliable labels to optimize the data adaptor with ZOO for label prediction. For data with low-confidence predictions, SODA encourages the adaptor to preserve data information to mitigate data corruption. Empirical results indicate that SODA can significantly enhance the performance of deployed models in the presence of distribution shifts without requiring access to model parameters.

SODA: Robust Training of Test-Time Data Adaptors
Zige Wang 1.2
Yonggang Zhang?
Zhen Fang² Bo Han
Long Lan
Wenjing Yang
1 School of Computer Science, Peking University University of Technology Sydney
Honn Kong Baptist University
"National University of Defense Technology
Abstract
Adapting models deployed to test distributions can mitigate the performance degra- dation caused by distribution shifts. However, privacy concerns may render model parameters inaccessible. One promising approach involves utilizing zeroth-order optimization (ZOO) train a data adaptor to adapt the test data to fit the deployed models. Nevertheless, the data adaptor trained with ZOO typically brings restricted improvements due to the potential corruption of data features caused by the data adaptor. To address this issue, we revisit 700 in the context of test-time data adaptation. We find that the issue directly stems from the unreliable estimation the gradients used to optimize the data adaptor, which is inherently due to the unreliable nature of the pseudo-labels assigned to the test data. Based this data adaptation (SODA) to improve
observation, we propose the performance of data adaptation. Specifically, SODA leverages high-confidence predicted labels as reliable labels to optimize the data adaptor with ZOO for la- bel prediction. For data with low-confidence predictions, SODA encourages the adaptor to preserve data information to mitigate data corruption. Empirical results indicate that SODA can significantly enhance the performance of deployed models in the presence of distribution shifts without requiring access to model parameters.
Introduction
Deep neural networks have emerged as a dominant tool in solving artificial intelligence problems due to their exceptional performance across various tasks [19 20 42]. thus being deployed various environments. However, in practice, these models typically suffer performance degradation due to distribution discrepancies between training and test data
351. To mitigate this issue, Test- Time Adaptation (TTA) is proposed as a promising solution, where unlabeled test data are leveraged modify the parameters of deployed models to alleviate performance degradation
In practice, the parameters of deployed models may be unmodifiable and inaccessible in many appli- cations due to intellectual property protection, misuse prevention, or privacy concerns in healthcare and finance [22] [46). The difficulty in modifying model parameters has hindered previous efforts aimed adapting deployed models to the test distribution [15 23 29
It is shown that training data adaptor to modify test data offers an alternative solution to mitigate the performance degradation caused by distribution discrepancy [18] 37.. However, the difficulty in accessing model parameters makes gradient computation a challenging task, hindering these data adaptation methods.
One straightforward approach involves utilizing zeroth-order optimization (ZOO) 225 train the data adaptor to adapt test data to fit deployed models. In particular, ZOO can be employed to estimate gradients for the optimization of data adaptors without modifying and accessing the parameters of deployed models. Therefore, test-time data adaptation with ZOO makes it possible to improve the performance of deployed models in many practical scenarios. However, introducing data adaptors
'Corresponding Author: Wenjing Yang(wenjing yang @nudt.edu.cn)
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

test data
Test
Corrupted
Adapted
averaged accuracy (%)
corrupted
50
40
30
adaptad data
20
10
CIFAR-10-C CIFAR-10C
brightness
Gaussian noise
snow
pixlelate
Demonstration of corrupted data and adapted data. The left part shows examples of the original test data from CIFAR- 10-C, the corrupted data generated by the data adaptor trained with unreliable pseudo-labels, and the adapted data generated by the data adaptor in SODA. The right part shows the corresponding accuracy of test data, corrupted data, and adapted data on CIFAR-10/100-C.
Figure I:
trained with ZOO brings limited improvement. To endow data adaptors with reliability in improving model performance, revisit ZOO in the context of test-time data adaptation. Training of the data adaptor depends heavily on the predicted labels. Thus, unreliable predicted labels will lead to unreliable gradient estimations in ZOO, which makes data features corrupted rather than adapted to deployed models. This is consistent with our observations as depicted in Figure !
Built upon the observation, propose pseudo-label-rouut data adaptation (SODA) enhance deployed models with inaccessible parameters. SODA robustifies the training of data adaptors by splitting the test dataset into two subsets. One subset contains data with high-confidence predictions and is regarded as a reliable dataset. The other contains the remaining data points, considered to be an unreliable subset. For data in the reliable dataset, SODA trains the data adaptor using ZOO in a supervised manner. In the meantime, SODA encourages the data adaptor to preserve input information in an unsupervised manner for data corruption mitigation over unreliable data. To verify the efficacy of SODA, we evaluate it on three widely used benchmark datasets under various settings. Our experimental results demonstrate that SODA can effectively mitigate the performance degradation of deployed models in the presence of distribution shifts.
To up, main contributions presented follows:
To enhance deployed models with unmodifiable and inaccessible parameters, we propose to shift from model adaptation to data adaptation, where zeroth-order optimization is employed to estimate gradients for the training of the data adaptor. Unfortunately, test-time data adaptation causes corruption of data features, leading limited performance improvements.
To exploit the potential of test-time data adaptation, we find that the limited improvement predicted labels used in ZOO. Thus, we propose
issue stems from the unreliable nature SODA robustify the training of test-time data adaptation, where the data adaptor is trained to preserve input information for data with unreliable predictions.
We verify the effectiveness of SODA under various settings. Our experimental results demonstrate that SODA can directly enhance the deployed model under distribution shifts without accessing and modifying the model parameters.
2 Related Work
Test-time adaptation. Test-time adaptation is machine learning technique that addresses the distribution shift problem using only unlabeled test data before making predictions. To mitigate the distribution discrepancy, most previous works adapt the pre-trained model test data by modifying

---

the full or part of the model parameters. Some advanced works 77 21] adapt the feature extractor to obtain more efficient feature representations of test data, while others [T replace the last linear classification layer to improve prediction over the extracted features. Batch normalization calibration [3 34 43 49 is also exploited to adjust the statistics and affine parameters in batch normalization layers. Unlike previous works, our work focuses on situations where model parameters are unmodifiable and adapts test data to the pre-trained model as an alternative solution.
451 modify or
White-box input adaptation. Recently, several works have put effort into input-level optimization in test-time adaptation by changing the input data or features. Auxiliary anto-enccders [22 18]. amplitude features and Fourier-style calibration [50], label--preeery
features along with a generative model [31], and generative diffusion model [TO] are utilized to achieve input-level adaptation. These works require either modification of the model training process or gradients from the deployed model at test time. contrast, our work focuses on test-time data adaptation without accessing the training process and gradients of the deployed model, making it more practical and broadly applicable.
Domain adaptation of black-box predictors. Domain Adaptation of Black-Box Predictors (DABP) is a subcategory of unsupervised domain adaptation [9 30 that solves a more restricted and practical application setting. Under this setting, the pre-trained model is treated as black box, and only the model's ' output is available during adaptation. Few works [22 32 46) have proposed solutions to this challenge using knowledge distillation to transfer knowledge from the black-box model to target models. In this work, we address the black-box setting in the context of test-time adaptation and propose to perform data adaptation without knowledge transfer. Besides, while previous DABP works require training on the entire unlabeled test dataset, our proposed method can also deal with online black-box settings where the test data arrive sequentially.
Zeroth-order optimization. Zeroth-order optimization (ZOO) [25] is family of optimization methods that do not require gradient information search for the optimal solution. Instead, explores the searching space using various techniques [6 [4] to estimate the optimization direction. Although it can be less efficient than first-order optimization methods, ZOO methods can be helpful in scenarios where gradient information is unavailable or expensive to compute. For instance, recent studies have applied ZOO to perform adversarial attacks to black-box neural networks (48). hyperparameter optimization in federated learning without gradient descent [5]], and transfer learning on black-box models [40. In this work, we propose to leverage 700 for test-time data adaptation on deployed models with inaccessible gradient information.
3 Methodology
We mainly focus on the C-way image classification task with a distribution shift between the training and test data, following previous works (26 39 43). Given deployed model M with inaccessible parameters, our purpose is to improve its prediction accuracy on unlabeled test data X = {xi, ). Since the parameters and inner structures of M are unknown, only the output prediction probabilities are available from M during the entire adaptation process. Namely, the proposed method pseudo- label-robust data adaptation (SODA) aims to adapt X to M without requiring access to the parameters of M. The overall framework of SODA is shown in Figure 2
3.1 Preliminary
Before elaborating on our proposed framework, we introduce zeroth-order optimization (ZOO). ZOO is a gradient- frreeenttfreeeeeee alternative of first-order optimization (FOO), e.g., SGD, SCD, and Adam. Most 700 methods follow the structure of FOO and consist of three fundamental steps (25:: gradient estimation, descent direction computation, and point updating. They utilize function-value- gradient estimations approximate the full or stochastic gradients computed in FOO.
One commonly used ZOO gradient estimation strategy is multi-point estimation 6 24). Given a continuously differentiable objective function f() on a d-dimensional variable 0 E Rd multi-point estimation computes directional derivative approximation as follows:
1 14
vel(0) ==
[1(0 + wuu)) - f(O))u:),
(I)
3

---

Reliable pseudo-iabel selection
(a) Before Adaptation
(c) Zeroth- Optimization
Deployed models
X
X1c
0
0
Data adaptor
200
Deployed models
Deployed models
Co
x
Eq. (1)
7C
Eq. V
6m
8
(b) During Adaptation
Figure 2: The overall framework of SODA. (a) Before adaptation, SODA first performs reliable pseudo-label selection according prediction confidence. (b) During adaptation, the data adaptor with parameter 0 is trained over the test data with reliable pseudo-labels using cross-entropy loss and those with unreliable pseudo-labels using mutual information maximization. The gradient is estimated using (c) zeroth-orde optimization.
where U1 .... Uy are q random direction vectors typically drawn from the standard multivariate normal distribution (0, I). and j is the smoothing parameter. On a mini-batch of data points X1,... X the estimated gradient ö is the average of the multi-point estimations to all data points Xi,..,
8 - + Σ (0:xx...
(2)
Various strategies are adopted to compute the descent direction. For zeroth-order stochastic gradient descent (ZO-SGD) utilized in our work, the descent direction is set the current gradient estimation 8. The point updating rule is the same in traditional SGD: with learning rate ?)
0 = njō
(3)
3.2 Zeroth-Order Optimization in Test-Time Data Adaptation
Let G with parameters & the data adaptor for test-time data adaptation. For each test data point x; (i I....,,)). n)), G transforms it to form the adapted data x for inference as follows:
x = ()
Ideally, the data adaptor G should be trained by minimizing the KI. divergence between the predicted probabilities of the adapted data and the true labels of test data. Typically, the training process requires back-pree poockpropaggting.. the gradients from the deployed model the data adaptor. A challenge arises as gradient computation is infeasible for the deployed model M with inaccessible parameters. In this regard, ZOO provides an effective approach to estimating gradients, as discussed in the previous section. Considering the parameters 0 of the data adaptor G as the variables to be optimized, utilizing ZOO in tist-time& data adaptation is to replace the objective function f(0) with the training objective function used to train the data adaptor G. Assuming that the true one-hot label of a test data point x; given, and replacing function (0) Eq. ( with divergence loss L(.,)) = (TI-)l-), the directional derivative approximation w.r.t. L(.,)) and (x, is
PoC.
[Cly., M G(x;; 0 + )m,)) C(y,, M G(x;; 0)))u}}
(5)
pm

---

In test-time adaptation, the true labels of test data are obviously unknown. A common strategy is to use the predicted pseudo-label y, as the substitute of the true label y,.
However, the pseudo-labels are unreliable due to the inaccurate model prediction under distribution shifts, causing the corrupted data features depicted in Figure [ Let d; denote the disturbance of pseudo-label yi, i.c., y; = o; + yi, and : = M Gxx;; 0) denote the predicted probability of the adapted data point x² = (x;;; 0), the KL divergence loss at test point x; becomes:
L. = Hy; σ:) + (Yi, P o, log P
(6)
where (ce('s)) is the eross-entropy loss. Then, replacing y; with y, in Eq. ()) the directional derivative approximation becomes
p log 0+au, uj. P;
Vol. VoLce pau
where VoLon py Σj [f P, ) - Ly P ) is the ideal directional derivative
approximation. The derivations of Eq. 6 and Eq. 7 are deferred to Appendix A. In Eq. 77. the last term is the disturbing term directly introduced by o, causing the difference between VaC and the unreliable directional derivative approximation Vec₁. VeL, further leads unreliable estimated gradients in Eq. which hinders the optimization of & and the training of G.
3.3 Pseudo- Larelo---bbl--Ree Robust Training
A direct strategy to alleviate the impact of the disturbing term in Eq. 7 is to select pseudo-labes labels with small a The selected pseudo-labels form reliable pseudo-label set Y, train the data adaptor in supervised manner. In particular, two basic criteria for reliable pseudo-label selection are adopted in SODA: 1) the prediction confidence should be higher than threshold т, indicating the selected pseudo-labels have small disturbances; 2) the number of selected reliable pscudo-labels for each class should be less than (1 p)n/C maintain the balance among classes, where p is the noise ratio, and C' is the number of classes. The test data points corresponding to the selected reliable pseudo-labels are considered as reliable data set X,, which is trained over with cross-entropy loss Loe and Y,.
To mitigate the data corruption over the remaining test data points X,, with unreliable pseudo-labels, SODA trains over them in an unsupervised manner. Following previous works [21 [2), mutual information maximization 4
a widely-used unsupervised loss that can encourage both global diversity and local certainty of model predictions by maximizing the mutual information between the input data sample and the predicted probabilities. Thus, it adopted to preserve input information in X,, as shown in Eq. (8), where x, = G(x;; 0) and p. M G(;;;).
36 41
C
Lim (x = Ex ex
Pik log Pial - Σ Expexe SeeexePul Pa log Exgexe Pik.
(8)
A==
k=1
3.4 Theoretical Analysis
To theoretically show the effectiveness of the proposed pseudo-label-rouut training strategy, we analyze the expected gradient estimation error [5124] in the training of test-time data adaptor with zeroth-order optimization. For simplicity, we consider the special case where the estimated gradient equals directional derivative approximation with mini-batch size equal to 1. The expected gradient estimation error Rx between the estimated gradient VoL, and the true gradient VoL; w.r.t. the whole test dataset X is:
Rx Ex [Ell VoC. VoL. [|a]]
Denoting h(x;) a, log P. in Eq. (6), the gradient of the KL. divergence loss is VoL. VoLce Voh. Accordingly, the estimated gradient of the KL divergence loss VoL. = VoLco + Vah. Then, before applying pseudo- leeudo-label-oobust data adaptation, the upper bound of expected gradient estimation error is:
Rx Ex [|| VoCco VoLce | +
Aah Voh [|a]]
(10)
5

---

Algorithm SODA framework
Input: test data X, data adaptor G with parameters 0, deployed model M, adaptation epochs N,
learning 7/, a,
STEP 1: Before adaptation:
Compute p M(X)
Select Y X, and X., as described in Section 3.3
STEP 2: During adaptation:
for to N do
for mini-batch X, (xb,, Xb2 Xb, in X do
Compute Yo (yb,, Vb2 Mbys... Fb. where ÿb. M(xx.)
and their corresponding data Xr, ybi, "1,,...,,,, Xr,, J. the remaining data as Xux {xu, Xugi Xm.. (Xrr U Xus Xb) Compute x9, and x using Eq. (5) for each data point X,, and X
Select Ynu = Yr,, Y, = (fn : Shgs
(xr,
Compute Loe for X [x x
using Yr.
Compute Cim for X = {xa, x x using Eq.
8
Compute estimated gradients dec and 8m w.r.t. Loe and Lim using Eq. and Eq. (2
0 < 0 (16mm ad.e)
end for end for
In pseudo-label-robut training strategy, the test dataset X separated into reliable set X, learned by cross-entropy loss with pseudo-labels, and unreliable set X,, learned by mutual information loss, the expected gradient estimation error w.r.t. X becomes:
Rx- Ex. E||
VeLco VoCco + Voh Vah [|]]] Ex B[I VoLim VoLim ||2]]- (11)
According to the previous study K] minimizing the cross entropy loss Leo pt) is equivalent to maximizing the mutual information Cim- Then, the upper bound of the expected gradient estimation error w.r.t. X is:
Rx Ex, [||| VoLc--Vole 12 Ell Voh Voh Ex E[ VoLc [2]] (12)
Comparing Eq. (TO) and Eq. (1).. the upper bound of the expected gradient estimation error is tightened after applying the pseudo-label-robus training strategy, justifying that the proposed pseudo- label-robust training strategy can alleviate the disturbance caused by the unreliable pseudo-labels in the gradient estimation in zeroth-order optimization of the test-time data adaptor.
3.5
SODA Framework
Up to this point, we have discussed zeroth-order optimization in test-time data adaptation and our proposed pseudo-labei-robust training strategy. The overall training objective is shown in Eq. 13
(X Y,) -im (X.) ) + aLee (X,, Y,),
(13)
where is the balancing hyperparameter between mutual information maximization loss Cin and cross entropy loss Lee. The training algorithm of SODA summarized Algorithm
4 Experiments
4.1 Experimental Setup
Datasets. We first evaluate our proposed framework SODA on two widely used -iiitii-iitiittiinn benchmarks, namely CIFAR-10- and CIFAR-100-C [4), each containing 10,000 CIFAR-10// 10/100 test images corrupted by 19 kinds corruptions with severity levels. We further evaluate SODA on large scale dataset ImageNet-C [14] with 50,000 ImageNet validation images corrupted by the same corruptions as 10/10-0/100-C We conduct our experiments on the highest level of each corruption and report the averaged accuracies over 19 corruptions.

---

Table Average accuracies (%) on CIFAR-IO-C (C10-C), CIFAR- CLFAR-IOO-C(CHOO 100-CCCCO0-CC and ImageNet-C (IN-C). FO Grad. means the requirement of first-order gradient from the target model or deployed model. Model Mod. means the requirement of modifying the parameters of deployed models. Distill. indicates methods using knowledge distillation to learn target models. DA indicates methods using test-time data adaptation. MA indicates methods using model adaptation.
Categories
Methods
FO Grad.
Model Mod.
C10-C
C100-C
IN-C
Deployed
72.39
41.41
31.36
DINE
x
73.86
40.52
Distill.
BETA
X
75.71
39.62
DA-PGD
X
X
24.63
4.15
14.39
DA-ZOO-Input
X
X
68.70
31.53
17.57
DA
DA-Direct
X
X
70.48
37.67
29.37
DA-PL
X
X
72.93
41.44
31.91
SODA (Ours)
X
X
82.55
52.41
42.14
SODA-R (Ours)
X
88.39
60.31
48.70
MA
MA-SO
86.54
62.02
56.90
Baselines. Deployed is the deployed model without adaptation. We compare our proposed SODA framework with two DABP baselines utilizing knowledge distillation. DINE (22 distills the knowl- edge from the deployed model to target model by minimizing the KL divergence between smoothed pseudo-labels and target model predictions. BETA [46] divides the target domain into easy- and sord-to-adapl subdomains, then mutually distills twin networks with weak-strong augmentation on two subdomains. We further implement four vanilla baselines of test-time data adaptation using ZOO. DA-Direct directly generates adapted data instead of perturbations using the same network structure and initial pseudo-labels. DA-PGD adopts PGD [27] to directly generate perturbations using estimated gradients computed by ZOO and the training objective of SODA. DA-ZO0-Input uses the same data adaptor and training objective as SODA, except that the variables optimized in ZOO are the input data of the deployed model, i.e., the output adapted data of the data adaptor. The gradients of data adaptor parameters are computed based the estimated gradients w.r.t. the adapted data. DA-PL trains the same data adaptor as SODA using initial pseudo-labels. Assuming gradient information is accessible, deployed model. To compare SODA with model adaptation, we implement MA-SO that modifies model parameters (except the last linear layer) using the same training objective SODA.
also implement SODA-R using first-order gradients computed from the
Deployed model settings. For all experiments regarding CIFAR- 10/100-C tasks, we adopt the CIFAR- 10/100 pretrained ResNet-50 [1]] model used in [26] and 38 as the deployed model. For experiments regarding the ImageNet-C task, we adopt the ImageNet pre-trained ResNet-50 model provided by Torch Vision 28] as the deployed model. Except for SODA-R and MA-SO, the deployed model is frozen with only output probabilities accessible. For SODA-R, the deployed model is also frozen, but gradients can be computed and back proo-ppop through the model. For MA-SO, the deployed model is set in training mode, and all parameters can be modified.
Implementation details. For SODA, the data adaptor uses a small network with two convolutional layers and an instance normalization layer in between to generate perturbations added to the original test data. For SODA-R in the CIFAR- 10/100-C tasks, two ResNet blocks as in 33 are inserted between the convolutional layers to form a larger data adaptor. For SODA-R the ImageNet-C task, one ResNet block and a couple of downsampling/upsamppll data adaptor structure is described in Appendix B.3. For all methods except DINE, BETA, DA-PGD, and SODA-R, the data adaptor/model is optimized using SGD with learning rate 1e-3, momentum 0.9, and weight decay= = le-5. For DINE and BETA, we follow the same settings as in [22 and [46), and the target models are both ResNet-50 initialized with ImageNet- from Torch Vision 28). For DA-PGD, the step size is also set to be 1e-3. SODA-R RR is optimized using Adam with learning rate 1e-3 and weight decay le-5. Batch size 256 is fixed for all methods. The number of training epochs = 150 for all baselines except DINE and BETA. For DINE and BETA, we train them for 120 epochs and fine-tune them for 30 epochs. For all methods using ZOO, the query number g = 5 for CIFAR- 10-C and ImageNet-C and 10 for CIFAR- 100-C, smoothing parameter
layers are inserted instead. Detailed
weights downloaded
7

---

Table Comparison of SODA using ZOO with different query numbers and SODA using first-order gradients. The network structure, training objective, and training strategy are the same. Averaged accuracies (%) over corruptions are reported.
Query Numbers
10
20
50
SODA-FO
CIFAR-1O-C 82,43
82.55
82.57
82.59
82.53 52.97
85.97
CIFAR-100-C 51.03
52.19
52.41
52.97
54.32
== le-3. All experiments are repeated with three random seeds. The code implementation can be found at https:// honn/ttlr-group/SSn
4.2 Experimental Results
Effectiveness of SODA. Table shows the accuracies on CIFAR-1O-C, CIFAR-100-C and ImageNet- C Caveraged over 19 corruptions. In the mostly restricted setting where first-order gradient computation and model modification are both not allowed, SODA improves the deployed model prediction accuracy by a large margin, 10% on CIFAR-IO-C, 11% on CIFAR- 10R-C and % on ImageNet-C. SODA also outperforms the DABP baselines on CIFARIO-C and CIFARI00-C tasks. Note that DINE and BETA are excluded in experiments on ImageNet-C because they are required to initialize their target models by ImageNet pre-trained weights, which violates the TTA setting. Especially on CIFAR-ION --0-C and ImageNet-C C task with much lower initial prediction accuracy, all baselines except MA-SO fail to improve, while SODA and SODA-R make significant improvement.
SODA v.S. model adaptation. Further relaxing the restriction on the deployed model, when parameters of the deployed model are frozen, but gradient computation is feasible, SODA-R achieves comparable accuracy with completely unrestricted MA-SO on EIFAAR-10-40 and even hetter accuracy on CIFAR-1O-C. It shows that data adaptation can be effective model adaptation for deployed models with inaccessible parameters. On ImageNet-C, SODA-R performs worse than MA-SO, suggesting that improvement still needed on large-scale datasets.
Effect of pseudo-label-robust training strategy in SODA. The failures of DA-Direct show that directly generating data using unreliable pseudo-labels fails to adapt data to the deployed model due to the corrupted data features. By adding perturbations to the original test data, DA-PL improves from DA-Direct small degree but still fails enhance the deployed model. With the same data adaptor as DA-PI., SODA can improve the model prediction to a large degree, indicating the effectiveness of the proposed peeudo-label-robuse training strategy.
Effect of data adaptor parameter ZOO in SODA. Both using a perturbation generation strategy, DA-PGD fails by directly generating perturbations using estimated gradients without learning data adaptor, and DA- DO-ZOO-Input achieves worse results by directly optimizing the model input using ZOO. With the same training objective, the significant improvements made by SODA indicate the effectiveness of the data adaptor parameter ZOO adopted in SODA.
4.3 Discussion
Effect of query number in ZOO. To analyze the relation between query number q and adaptation accuracy in SODA, we conduct experiments with g E {2, 5, 10, 20, 50). The results in Table show that SODA not sensitive to query number used ZOO. Especially on CIFAR-10-C. only a slight performance drop is observed when g 2. On CIFAR-100- 100-C, larger query number contributes to accuracy improvement more observably, but when q > 10. the contribution becomes less efficient, as the computation cost is also increased. Balancing the trade-off between accuracy and computation costs, we finally choose = for CIFAR-IO-C -O-C and 10 for C00-R-100-C.
Zeroth-order optimization v.s. first-order optimization. To better illustrate the effect of ZOO, we implement a comparing baseline SODA-FO that shares the same setting with SODA but uses the first-order gradients back proopagatt from the deployed model. Note that SODA-FO and SODA-R differ in data adaptor network structure, optimizer, and other strategies as discussed in Appendix C.I. As shown in Table 2 although SODA does not achieve the same accuracies as SODA-FO due to the unavoidable gradient estimation error, it still indicates competitive performance when gradient computation is infeasible. In Figure SODA using ZOO has the slowest convergence speeds
8

---

accuracy on gaussaan naise
Deployed
S00A
80
averaged
500k
5000.r0 500A.a
5006 A
100
0.50
1000
5000
19460
points
(a)
(b)
Figure
Comparison of SODA trained over different numbers of test samples.
4:
Figure 3: Accuracy convergence on Gaussian noise in (a) CIFAR-
10-C and (b) CIFAR-100-C.
Table 3:
Comparison of SODA and SODA-R using data adaptor with different numbers of ResNet blocks. Averaged accuracies (%) over 19 corruptions on CIFAR-10-C are reported.
Table 4: Average accuracies (%) on CIFAR-IO-C (C10-C) and CIFAR-100-C (C100-C) under online setting with difference batch size (epochs = 10). D is the deployed model, BS batch size.
#blocks
3
Methods
SODA-O
SODA 82.55
81.91
80.56
78.65
BS
32
64
128
256
SODA-R
86.07
88.07
88.39
88.51
C10-C
72.39
78.01
78.66
78.79
77.03
C100-C
41.41
45.82
47.11
47.26
45.93
and lowest accuracies both datasets. SODA-FO has similar convergence speeds SODA-R but converges to lower accuracies, indicating that the strategies used in SODA-R with first-order optimization, i.e., deeper network and Adam optimizer, boost the training of the data adaptor.
Effect of data adaptor network complexity. We also explore the effect data adaptor network complexity on SODA and SODA-R. To increase network complexity, we change the number of ResNet blocks the data adaptor. Results data adaptor with (0, 1, 2, 3) ResNet blocks are reported in Table 3 For SODA, accuracy decreases network complexity increases, indicating that complex networks hinder data adaptation using ZOO, However, the accuracy of SODA-R increases along with network complexity. This contrast illustrates that a more complex data adaptor achieve higher accuracy but restricted and even encumbered by zeroth-order gradient estimation.
SODA with fewer test data points. We further explore the effectiveness of SODA with fewer test data points. We randomly choose (50, 100,,00, 500, 1000, 5000, 10000} test data points evenly distributed across 10 classes in CIFAR-10-C for each corruption smaller test datasets. The averaged accuracies are reported in Figure 4 The performance of SODA is better when training data adaptor over more test data points. Nevertheless, training over 5,000 data points achieves comparable accuracy with training over 10,000 data points, showing that SODA does not require an extremely large number of data points to achieve good performance. Besides. SODA can still improve with less than 500 test data points, providing promising insights for test-time data adaption with smaller test datasets.
Hyperparameter sensitivity. We conduct sensitivity analysis regarding the noise ratio the confidence threshold r and the balancing parameter a. The detailed results illustrated Appendix C.3 show that SODA is robust to different combinations of hyperparameters, but adapting an adapted threshold instead fixed threshold might further improve the performance SODA.
4.4 SODA for Online Test-Time Adaptation
More practically, test data is not entirely available but arrives sequentially, i.c., online test-time adaptation. Hence, we further implement SODA-O as variant of SODA under online settings. Given mini-batches of test data arrived sequentially {X1...., X,). SODA-O adapts mini-batch at time before processing the next mini-batch with knowledge accumulated from the previous mini-batches.
The main difference between SODA-O and SODA lies in the reliable pseudo-label selection. Without access the test data before adaptation, SODA-O maintains an ordered queue Q with maximum size S to store the selected reliable pacudo-labels Y, and their corresponding data points X,.. Specifically,

---

Table 5: Average accuracies (%) on CIFAR- 10-C and CIFAR-100-C under online setting with different number of epochs per batch (batch size 256).
Methods
Deployed
SODA-O
SODA
Epochs/Batch
5
10
30
50
100
150
150*
CIFAR-10-C
72.39
75.22
77.03
79.63
80.38
81.33
81.71
82.55
CIFAR-100-C 41.41
43.59
45.81
48.56
49.26
50.04
50.12
52.41
#SODA trained over the entire test dataset for 150 epochs
for a mini-batch X arrives at time t, reliable pseudo-labels Yr, with prediction confidences higher than 7 are selected and pushed into Q along with their corresponding test data points X,,- To maintain the class balance in Q, the pseudo-labels with the smallest confidence for class i will be popped out once the number of pseudo-labels for k in Q is larger than S/C. Then, the remaining data points X., in X, are considered as X all pseudo-labels and data points stored in Q are considered as Y,, r. and X X and X", form small test dataset to train the data adaptor as in SODA, i.e. Step 2 in Algorithm After adaptation, the inference of X2 is given by the current data adaptor and the deployed model. Note that the optimization in SODA-O is not repeated after reaching the entire test dataset but only repeats for the current test data batch and the cached queue. During the adaptation of the current test data batch, the previous data batches are no longer available except for those saved in the queue. The data adaptor, hyperparameter setting, and training strategy of SODA-O are the same as SODA. The queue size S = 1,000 for both CIFAR-10-C and CIFAR-100-C.
Effect of epochs per mini-batch. We fix batch size = 256 and conduct experiments with different numbers of epochs per mini-batch, i.e., (5. 10, 30, 50, 100, 150) epochs/batch. As shown in Table 5 SODA-O is effective under online setting, As epochs/batch increase, the accuracy of SODA-O also increases and approaches the accuracy of SODA. But more training epochs means more processing trade-off. With 10 epochs/batch, SODA-O can
time for each mini-batch leading still improve the deployed model by 5% and 4% on CIFAR- 10-C and
Effect of batch size. Fixing epochs/batch = 10, we also conduct experiments with different batch sizes, [32, 64, 128, 256). The averaged accuracies are shown in Table The results show that the performance of SODA-O is stable across different batch sizes. The best performance is achieved with batch size= = 128. Only a slight performance drop is observed when batch size is reduced to 32, indicating that SODA-O can handle relatively small batch size.
5 Limitations
While providing valuable insights into robust test-time data adaptation for deployed models with inaccessible parameters, it is also essential to consider the limitations that may have affected the efficiency and effectiveness of our proposed framework. The main limitation of SODA is the time consumption brought by i) multiple queries required for gradient estimation in ZOO and ii) multiple passes required for training the random initialized data adaptor. In the future, one solution to alleviate this issue could be leveraging ZOO methods with higher query efficiency and less estimation error. Moreover, it is unclear whether SODA designed for OOD generalization can benefit OOD detection 8 44 Thus, leave the exploration for OOD detection as our future work.
6
Conclusion
This paper focuses on two major challenges in adapting deployed machine learning models with inaccessible parameters test-time: unmodifiable model parameters and infeasible gradient com- putation. Without modifying the model parameters, a data adaptor is adopted to adapt test data to the deployed model. Zeroth-order optimization is further leveraged train the data adaptor with estimated gradients. Revisiting ZOO in test-time data adaptation, we discover that the unreliable gradient estimation in ZOO is due to the unreliable pseudo-labels assigned to test data. The proposed pseudo-label--ohne data adaptation (SODA) addresses this issue with reliable pseudo-label selection and input information maximization. Our experiments on three widely used -ut-r--diitiibuutin benchmarks demonstrate the effectiveness of SODA in both offline and online settings.
10

---

Acknowledgments and Disclosure of Funding
This work was partially supported by the National Natural Science Foundation of China (No. 62372459, No. 62376282).
References
[1] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk mini- mization. arXiv preprint arXiv:1907.022 2019.
[2] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to sume-supe learning. In Advances in Neural Information Processing Systems, 2019.
[3] Malik Boudiaf, Jérôme Rony, Imtiaz Masud Ziko, Eric Granger, Marco Pedersoli, Pablo Piantanida, and Ismail Ben Ayed. A unifying mutual information view of metric learning: cross-cntropy vs. pairwise losses. In European Conference on Computer Vision, 2020.
[4] John Bridle, Anthony Heading, and David MacKay. Unsupervised classifiers, mutual informa- tion and" phantom targets. In Advances in Neural Information Processing Systems, 1991.
[5] Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In ACM Workshop on Artificial Intelligence and Security, 2017.
[6] John C Duchi, Michael Jordan, Martin J Wainwright, and Andre Wibisono. Optimal rates for zero-order convex optimization: The power of two function evaluations. IEEE Transactions on Information Theory, (1())27788-22
2015.
[7] Cian Eastwood, Ian Mason, Christopher KI Williams, and Bernhard Schölkopf. Source-free adaptation to measurement shift via bottom-up feature restoration. In International Conference on Learning Representations, 2022.
[8] Zhen Fang, Yixuan Li, Jie Lu, Jiahua Dong, Bo Han, and Feng Liu. Is out-of-distribution detection learnable? In Advances in Neural Information Processing Systems, 2022.
[9] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International Conference on Machine Learning. 2015.
[10] Jin Gao, Jialing Zhang, Xihui Liu, Trevor Darrell, Evan Shelhamer, and Dequan Wang. Back the source: Diffusion-driven test-time adaptation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
[11] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2016.
[12] Yufan He, Aaron Carass, Lianrui Zuo, Blake E Dewey, and Jerry L Prince. Autoencoder based self-supervised test-time adaptation for medical image analysis. Medical image analysis, 72: 12:102136, 2021.
[13] Dan Hendrycks, Steven Basart, Norman MU, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In International Conference Computer Vision, 2021.
[14] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations, 2019.
[15] Jiaxing Huang, Dayan Guan, Aoran Xiao, and Shijian Lu. Model adaptation: Historical contrastive learning for unsupervised domain adaptation without source data. In Advances in Neural Information Processing Systems, 2021.
[16] Yusuke Iwasawa and Yutaka Matsuo. Test-time classifier adjustment module for model-agnostic domain generalization. In Advances in Neural Information Processing Systems, 2021.
[17] Minguk Jang and Sae- information. In International Conference on Learning Representations, 2023.
Sae-Young Young Chung. Test-time adaptation via self- self-training with nearest neighbor
11

---

[18] Neerav Karani, Ertunc Erdil, Krishna Chaitanya, and Ender Konukoglu. Test-time adaptable neural networks for robust medical image segmentation. Medical Image Analysis, 68:101901 2021.
[19] Alex Krizhevsky, Πla Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84-999) 2017.
[20] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436-444,
2015.
[21] Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning, 2020.
[22] Jian Liang, Dapeng Hu, Jiashi Feng, and Ran He. Dine: Domain adaptation from single and multiple black-box predictors. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
[23] Hyesu Lim, Byconggeun Kim, Jaegul Choo, and Sungha Choi. Tin: A domain-shift aware batch normalization in test-time adaptation. In International Conference on Learning Representations 2023.
[24] Sijia Liu, Jie Chen, Pin-Yu Chen, and Alfred Hero. Zeroth-order online alternating direction method of multipliers: Convergence analysis and applications. In International Conference on Artificial Intelligence and Statistics, 2018.
[25] Sijia Liu, Pin- Chen, Bhavya Kailkhura, Gaoyuan Zhang, Alfred O Hero III, and Pramod Varshney. A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications. IEEE Signal Processing Magazine, 37(55433 2020.
[26] Yuejiang Liu, Parth Kothari, Bastien Van Delft, Baptiste Bellot-Gurlet, Taylor Mordan, and Alexandre Alahi. Ti+++: When does self-supervised test-time training fail or thrive? In Advances in Neural Information Processing Systems, 2021.
[27] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference Learning Representations. 2018
[28] TorchVision Vision maintainers and contributors. Torchvision: Pytorch's computer vision library.
https://gt www.ggtthhu....e htt.ub...com/pttooc//iiiisooooot 2016.
[29] Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. In International Conference on Learning Representations, 2023.
[30] Sinno Jialin Pan and Qiang Yang. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 13110):345--335, 2010.
[31] Prashant Pandey, Mrigank Raman, Sumanth Varambally, and Prathosh Ap. Generalization on unseen domains via inference- time label- pressrring target projections. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
[32] Qucheng Peng, Zhengming Ding, Lingjuan Lyu, Lichao Sun, and Chen Chen. Toward bet- ter target representation for source-free and black-box domain adaptation. arXiv preprint arXiv:2208. 10531, 2022.
[33] Omid Poursaeed, Isay Katsman, Bicheng Gao, and Serge Belongie. Generative adversarial perturbations. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.
[34] Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. Improving robustness against common corruptions by covariate shift adaptation. In Advances in Neural Information Processing Systems, 2020.
[35] Zheyan Shen, Jiashuo Liu, Yue He, Xingxuan Zhang, Renzhe Xu, Han Yu, and Peng Cui. Towards -fitt--ditti generalization: A survey. arXiv preprint 2.iv:2108..3624 2021.
[36] Yuan Shi and Fei Sha. Information information-theorrttiall learning of discriminative clusters for unsuper- vised domain adaptation. In International Conference Machine Learning, 2012.
12

---

[37] Manli Shu, Weili Nie. De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test-time prompt tuning for zero-shot generalization in vision-language models. In Advances in Neural Information Processing Systems, 2022.
[38] Yongyi Su, Xun Xu, and Kui Jia. Revisiting realistic test-time training: Sequential inference and adaptation by anchored clustering. In Advances Neural Information Processing Systems, 2022.
[39] Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexci Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. In International Conference on Machine Learning, 2020.
[40] Yun-Yur Yun Tsai, Pin- Yu Chen, and Tsung-Yi Yi Ho. Transfer learning without knowing: Repro- gramming black-box machine learning models with scarce data and limited resources. In International Conference on Machine Learning. 2020.
[41] Michael Tschannen, Josip Djolonga, Paul Rubenstein. Sylvain Gelly, and Mario Lucic. On mutual information maximization for representation learning. In International Conference on Learning Representationss, 2020.
[42] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Informa- tion Processing Systems, 2017.
[43] Dequan Wang. Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations, 2020.
[44] Qizhou Wang, Feng Liu, Yonggang Zhang, Jing Zhang, Chen Gong, Tongliang Liu, and Bo Han. Watermarking for diittii-iitiitttoon detection. In Advances Neural Information Processing Systems, 2022.
[45] Zehao Xiao, Xiantong Zhen, Ling Shao, and Cees GM Snoek. Learning to generalize across domains on single test samples. In International Conference on Learning Representations, 2021.
[46] Jianfei Yang. Xiangyu Peng, Kai Wang. Zheng Zhu, Jiashi Feng, Lihua Xie, and Yang You. Divide to adapt: Mitigating confirmation bias for domain adaptation of black-box predictors. In International Conference on Learning Representations, 2023.
[47] Yonggang Zhang, Mingming Gong, Tongliang Liu, Gang Niu, Xinmei Tian, Bo Han, Bernhard Schölkopf. and Kun Zhang. Adversarial robustness through the lens of causality. In International Conference on Learning Representations, 2022.
[48] Yonggang Zhang, Ya Li, Tongliang Liu, and Xinmei Tian. Dual-path distillation: A unified framework to improve black-box box attacks. In International Conference on Machine Learning. 2020.
[49] Bowen Zhao, Chen Chen, and Shu-Tao Xia. Delta: degradation-free fully test-time adaptation. In International Conference on Learning Representations, 2023.
[50] Xingchen Zhao, Chang Liu, Anthony Sicilia, Scong Jae Hwang, and Yun Fu. Test-time fourier style calibration for domain generalization. In International Joint Conference on Artificial Intelligence, 2022.
[51] Yi Zhou, Parikshit Ram, Theodoros Salonidis, Nathalie Baracaldo, Horst Samulowitz, and Heiko Ludwig. Flora: Single-shot hyper-parameter optimization for federated learning. arXiv preprint arXiv:2 02.05224 2021.
13

---

Appendix
A Derivation of Directional Derivative Approximation in SODA
given deployed model M, the ideal objective function of training the data adaptor G with parameters 0 in SODA is the KI. divergence between the predicted probability P M (x;; 0) of the adapted data point x Gx;;; 0) and the true label y; of the original data point X;- Because y, is not available at test time, pseudo-label y; predicted by M is adopted as substitute
In Section 3.2
yf. Due to the inaccurate model prediction under distribution shifts, there a disturbance o in
Yi compared to yı, i.e. y; = o; yz. Hence, the KL divergence loss [(,,)) == KL(-Il-) at test data
point x; is
y:
L: = KL(V:ll!!) = 8i log p
(y: + o.) Yı + o. p
(14)
(y, + o., log( + o;) y, log p. - o, log p
-yH +,, + Le (y., p. o; log
where Lye(y; p is the cross entropy loss between y, and p.. Because the gradient information inaccessible from the deployed model, zeroth-order optimization (ZOO) utilized to estimate
gradients for the training of the data adaptor in SODA. To do this, the objective function f(0) in
Eq. I is replaced with the training objective function L; in test-time data adaptation. Denote L as the KL divergence loss computed by data adaptor with parameters the directional derivative approximation of ZOO is
VoL. nqq
1
Σ
(KL()) - (L(x)(1p)))
j=1
[y buthu)) o, log puu (Cooly, p.) o. log (:)))
pn
Σ [[Cooyy, @ - Lee p.]) + μq j=l
Σ
[0, p o, loo.p. 0+uu [,]
pq 1=1
+
log
0+uu Uj
3=1
P;
(15)
where VoLos 1 approximation.
Σラ
[foe (y:. P. p.tm)))
Lee (vi,, is the ideal directional derivative
=1
B Implementation Details
B.1 Implementation details of DINE and BETA
The implementations of DINE and BETA CIFAR-10-C and CIFAR-100-C kept the same, following their original work 22 and 46 For DINE, the momentum hyperparameter = 0.6 and the Mixup balancing hyperparameter
For BETA, 0.8 for domain division, a 1.0 for Mixup, Amse = O, sharpening factor T = 0.5, and adversarial regularizer of DINE and BETA are both SGD with learning rate 0.001 for target network backbones and 0.01 for MLP classifiers. momentum 0.9 and weight decay = 1e-3 are also adopted.
0.1. The training strategy
B.2 Software and hardware
In our paper, all models are implemented using PyTorch 1.13.1. The ImageNet pre-trained weights used DINE and BETA are downloaded from Torco 0.14.1. The experiments are conducted using NVIDIA A100-PCIE-40GB GPU with CUDA 11.7.
14

---

B.3 Network structure of data adaptor
Figure [] shows the network structure of the data adaptor used in our experiments. The basic structure of the data adaptor consists of two convolutional layers and an instance normalization layer in between. Multiple ResNet blocks and downsamping/upsampa convolutional layers to form deeper network as in [33] For all methods except DA-Direct, the adapted data generated by treating the network output as perturbation and adding it the original data.
layers can be inserted into the
X
x
ResNet
ResNet
Figure 5: Network structure of data adaptor. The green block is the convolutional layer, the blue block is the instance normalization layer, the white black is the dropout layer, and the orange block is the downsaapiing/upsamping layer. The dashed blocks can be added removed to form different network structures. The red dashed line means the network output is added to the original data to generate the adapted data.
C Additional Analysis
C.1 Discussion about SODA and SODA-R
Compared with SODA. SODA-R uses computed first-order gradients and adopts several techniques to improve the performance, i.e., deeper data adaptor with ResNet blocks, Adam optimizer, perturbation regularization, and dropout. The effect of network complexity has already been discussed in Section 4.3. In this subsection, we first introduce the perturbation regularization used in SODA-R, then evaluate effect of perturbation regularization, different optimizers, and dropout on SODA and SODA-R.
C.1.1 Perturbation regularization in SODA-R
In SODA and SODA-R, the adapted data is computed by perturbing the original data with generated perturbation. To further restrict the impact of generated perturbations on data X, (Xry; Xr2 ),,, and X,, = (Xay: Xug; ..., Xu, perturbation regularization with i norm used:
let x6, and x be the corresponding adapted data of x,, and X,..
R(X) = Ex. | x,,11 + Ex, x 1|x
x., l1:
(16)
First-order gradients of the perturbation regularization are directly computed and back-propagated through the data adaptor. Hence, the training objective of SODA-R becomes:
(X, Y,) Lim (X.)) ). + aee (x,, Y,) (R(X),
(17)
where 8 is the weight of perturbation regularization and is set be 0.005 for CIFAR-10-C and 0.01 for CIFAR-100-C.
C.1.2 Evaluation of perturbation regularization in SODA and SODA-R
We evaluate the effect of perturbation regularization in SODA and SODA-R on CIFAR-10-C and
CIFAR-100-C Except for the perturbation regularization term in the training objective, all other
settings are kept the same in the main experiments. The results are shown in Table T and Table 7
15

---

It shows that perturbation regularization can improve the performance of SODA-R using first-order optimization, especially on CIFAR-100-C However, largely hinders the performance SODA using zeroth-order optimization. The computed first-order gradients of perturbation regularization are more accurate than the estimated zeroth-order gradients of the main training objective. Thus, the data adaptor tends to optimize the perturbation regularizan laizzatio term first, resulting in perturbations with too small norms. The perturbations with too small norms do not have enough ability to modify the test data, which might be the reason for the worse performance achieved by SODA with perturbation regularization. One possible solution to this problem could be treating perturbation regularization as an optimization constraint and using constrained ZOO methods to train the data adaptor.
Table 6: Comparison of SODA and SODA-R with and without perturbation regularization on CIFAR-IO-C. 3 0.005 in experiments w/ reg- ularization.
Table 7: Comparison of SODA and SODA-R with and without perturbation regularization on CIFAR-100-C. 3 = 0.01 in experiments w/ regularization.
Methods
SODA
SODA-R
Methods
SODA
SODA-R
w/ regularization
73.40
88.39
w/ regularization
42.27
60.31
w/o regularization
82.55
87.96
w/o regularization
52.41
58.11
C.1.3 Evaluation of optimizers in SODA and SODA-R
We evaluate the effect of optimizers used in SODA and SODA-R on CIFAR-IO-C and CIFAR-100 Except for the optimizer used to train the data adaptor, all other settings are kept the same as in the main experiments. The results are shown in Table S and Table On CIFAR-IO-C, SODA trained by SGD and Adam achieve almost the same accuracy, while SODA-R trained by Adam achieves 3.3% higher accuracy than SGD. On CIFAR-100-C, SODA-R trained by Adam still outperforms SODA-R trained by SGD, but SODA trained by Adam achieves by SGD. It shows that Adam optimizer can improve the training of data adaptors using first-order gradients but fails when using the estimated zeroth-order gradients.
worse accuracy than SODA trained
Table 8: Comparing of SODA and SODA-R using SGD and Adam optimizer CIFAR-10- C.
Table 9: Comparison of SODA and SODA-R using SGD and Adam optimizer CIFAR- 100- C.
Methods
SODA
SODA-R
Methods
SODA
SODA-R
SGD
82.55
84.95
SGD
52.41
58.32
Adam
82.75
88.39
Adam
49.75
60.31
C.1.4 Evaluation of dropout in SODA and SODA-R
We also evaluate the effect of dropout on SODA and SODA-R. As depicted in Figure 5 dropout layer can be inserted into the ResNet block. We conduct experiments using data adaptors with and SODA-
without dropout layers for SODA and SODA-R. To keep the same network structure R, the data adaptor used in SODA also has 2 ResNet blocks. The dropout ratio is set to be 0.5. All other settings are kept the same in the main experiments. Table I0 and Table I show the results on CIFAR-IO-C and improve the accuracy by 0.7% on CIFAR-IO-C and 2% on CIFAR-100-C. However, for SODA, adding dropout layers extremely hinders the performance, especially on CIFAR- This contrast indicates that dropout harms data adaptor optimized using estimated zeroth-orded gradients while positively affecting data adaptor optimized using computed first-order gradients. The reason might be that the extra randomness introduced by dropout increases the difficulty of gradient estimation in zeroth-order optimization. Note that the accuracy of SODA using data adaptor with 2 ResNet blocks on CIFAR- 10-C is worse than that using data adaptor with ResNet blocks, which consistent with the results on CIFAR- 10-C shown in Table
respectively. For SODA-R, adding dropout layers can
16

---

Table 10: Comparing of SODA and SODA-R with and without dropout layers on CIFAR-10- C.
Table 11: Comparison of SODA and SODA-R
with and without dropout layers on CIFAR-100
Methods
SODA
SODA-R
Methods
SODA
SODA-R
w/ dropout
32.19
88.39
w/ dropout
5.47
60.31
w/o dropout
80.56
87.54
w/o dropout
43.96
58.27
To sum up. compared with SODA using zeroth-order optimization, SODA-R uses first-order op- timization and adopts deeper network structure, perturbation regularization. Adam optimizer, and dropout to improve the performance. However, these techniques cannot improve even hinder the performance of SODA. This comparison shows that the common boosting strategies used in first-order optimization cannot be directly applied zeroth-order optimization, leading the limited performance of methods using zeroth-order optimization.
C.2 Convergence of SODA
the convergence speeds of SODA on CIFAR-1O-C and CIFAR-100-4 are slower than SODA-FO and SODA-R and do achieve complete convergence after training with 150 epochs. We further train SODA on Gaussian noise CIFAR-10-4 10-C and CIFAR-100-C for 300 epochs to show the complete convergence of SODA depicted in Figure 6 With more training epochs, SODA can achieve higher accuracies on both datasets. However, training with more epochs means more adaptation processing time resource efficiency, we only report the accuracies achieved at 150 epochs our main experiments, which already improves the deployed model large margin. If computing time and resources not restricted, SODA can further improve the deployed model for higher accuracy.
In Figure 3
computing resources with parallel computation. For time and
accuracy on gausslan san noise (%)
accuracy on gaussian noise (%)
40
70
3.5
65
30
600
55
epoch
epoch
(a)
(b)
Figure Accuracy convergence on Gaussian noise in (a) CIFAR-1O-C and (b) CIFAR-100-C for 300 epochs.
C.3 Hyperparameter analysis of reliable pseudo-label selection
We evaluate the hyperparameters reliable pseudo-label selection, namely the confidence threshold the noise ratio p. and the balancing parameter т and p controls the number of selected reliable lseudo-labels. With lower and lower p. the number of selected reliable pseudo-labed increases. We evaluate 7 in {0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}. and in {0.1, 0.3, 0.5, 0.7, 0.9). Note that number of selected pseudo-labels is not equal (1 p)n,, where number of test data points, because the pseudo-labels are not evenly distributed across classes as depicted in Figure Sa The inaccurate model prediction tends bias towards few classes, leading to more pscudo-labels belonging those classes. controls the balance between the supervised training
when 7 = 0,
the total
17

---

objective Lce and the unsupervised training objective Lim. We evaluate a in {0.01, 0.001, 0.0001 The results of SODA with different sets of hyperparameters on CIFAR-IO-C Gaussian noise level comrpption are shown in Figure The performance of SODA is stable across different hyperparameter settings. A common trend among different a that accuracy tends increase when т and p decrease, i.e., the top-right corner of each figure. This trend shows that the performance of the data adaptor can improved using more selected pseudo-labels, which further indicates the reliability of the selected pseudo-labels. There is mild tendency of performance drop in the overall performance of SODA when unsupervised learning test points with unreliable pseudo-labels is overwhelmed by supervised learning of reliable pseudo-labels with larger a, indicating that learning with unreliable pseudo-labels also has a contribution to the performance of SODA. To balance the supervised and unsupervised learning terms, finally choose 0.0001 in main experiments. Although better performance can be achieved by carefully fine-tuning and with a validation set show the general performance of SODA and select the most reliable pseudo-labels for different corruptions, we set 7 = 0.9 and 0.9 for CIFAR- 10FAR-10/100 10/100-C tasks and r = 0.1 and 0.9 for ImageNet-C task in main experiments without elaborated hyperparameter fine-tuning.
test points
0.01
[= 0.001
0.0001
74.00
72.27
73.30
73.33
73:16
73:26
73.50
73.30
73:36
73.75
72.27
73.30
73.53
73.16
73.26
72.35
73.50
73.30
73.58
72.27
73:15
73.49
73.30
73.33
73.16
73:26
73.50
73.30
73.36
73.50
73.32
73.25
72.27
72.93
73.29
73.19
73.00
72.85
73.22
73.48
73.44
73.25
0-4
72.35
73.30
73.75
73.45
73.24
73.00
-2.227
72.91
73.15
73.25
72.92
72.65
73.15
73.31
73.10
73.52
77.55
73.23
73.22
73.21
73.40
72.50
72.27
72.71
73.18
73.01
72.57
0.
72.65
73.08
73.09
73.01
73.00
72.55
72.97
73.24
73.04
73.23
09
0.8
0.9
0.5
0.3
p
Figure 7: Evaluation of reliable pseudo-label selecting hyperparameters on CIFAR-IO-C Gaussian noise corruption level 5. Numbers are prediction accuracies (%) after adaptation.
C.4 Evaluation of queue size in SODA-O
We evaluate the effect of queue size in SODA-O. A larger queue size means more past reliable pseudo-labels and their corresponding test data points are stored and used in the adaptation pro- cess of the current mini-batch. Fixing batch size = 128, we conduct experiments on queue size (500, 1000, 2000, 3000), and the results are shown in Table 12 The performance of SODA-O is stable across different queue sizes, especially when the queue size is smaller. When queue size increases, the ratio of reliable pseudo-labels used to train the data adaptor for the current mini-batch also increases. It makes the training the data adaptor more biased towards the supervised training with the reliable pseudo-labels. Thus, the mild performance drop observed along with the larger queue size might indicate that the reliable pseudo-label. still have disturbance, and the unsupervised training of data points with unreliable pseudo-labels is useful alleviate the negative effect caused by the remaining disturbance.
C.5 Results on challenging Office-Home tasks
We conduct experiments on three challenging Office-Home tasks to show the effectiveness of SODA and SODA-R on different kinds of distribution shifts. For SODA, the network structure of the
18

---

Table 12: Comparison SODA-O with different queue sizes. Averaged accuracies (%) over 19
corruptions are reported.
Queue Size
500
1000
2000
3000
CIFAR-1O-C
78.78
78.79
78.22
77.73
CIFAR-100-C
47.18
47.21
46.67
45.63
data adaptor is the same that in CIFAR-10/100-4 tasks. For SODA-R, a deeper data adaptor is adopted with 2 wwwnsaapiin//ussaappingg layers and ResNet blocks. Both SODA and SODA-R are trained using SGD with learning rate 1e-3, momentum = 0.9, and weight decay lc-5 for 60 epochs. warmup process is used first train the data adaptor with pseudo-label supervision for 10 epochs. Following DINE [22] and BETA 46), MixMatch [2] and EMA pseudo-label updating are also adopted. The weights cross- moos-ey loss and mutual information loss are and 0.01, respectively. The confidence threshold r is set to be 0.5. The noise ratio p is set to be O. The query number is set to for SODA. The results are listed Table 13] The results show that SODA and SODA-R can improve the deployed model in challenging Office-Home tasks with initial prediction accuracies lower than 50%
be 0.
Table 13:
Accuracies (%) three challenging Office-Home domain adaption tasks. FO Grad. means the requirement of first-order gradient from the target model or deployed model. Model Mod. means the requirement of modifying the parameters of deployed models.
Methods FO Grad.
Model Mod.
Art->Clipart
Product->Clipart
Realworld->Clipat
Deployed SODA
44.67
40.53
47.12
x
X
46.53
41.47
48.71
SODA-R
/
X
50.45
45.20
53.22
D Qualitative Evaluation of SODA
D.1 T-SNE Visualization SODA features
To qualitatively evaluate the performance of SODA, we use T-SNE visualize the feature embeddings of SODA, i.e., the input features of the last classification layer in the deployed model before and after adaptation Figure According the visualization results, the feature embeddings are much more separated apart between classes after adaptation, showing the effectiveness of SODA.
-20
-10
(a) Before adaptation
After adaptation
Figure 8: T-SNE visualization of SODA feature embeddings on CIFAR-10-C pixelate corruption level
19

---

D.2 Examples of Adapted Data
Figure shows examples of test and adapted data using SODA for 19 corruptions in CIFAR-IO-C. Comparing original data without corruption, test data before adaptation, and adapted data after adaptation, is obvious that the adapted data looks closer to the original data than the corresponding test data. This observation is consistent with the improved prediction accuracy using SODA and further illustrates that the distribution shifts between the test data and the training data are alleviated after applying SODA to test data. It also indicates that SODA adapts the test data to the deployed model by modifying them to "look like" the training data. Then, the distribution shifts between the test data and the training data are mitigated, leading to improved prediction of the deployed model.
E Detailed Results
There are 19 corruptions CIFAR-1O-C, CIFAR-10O-C and ImageNet-C: Gaussian noise (GN), shot noise (ShN), impulse noise (IN), speckle noise (SpN), defocus blur (DB), glass blur (GIB), motion blur (MB), zoom blur (ZB), Gaussian blur (GaB), snow (SW), frost (FR), fog (FG), brightness (BR), contrast (CT), elastic transform (ET), pixelate (PX), jpeg compression (JC), spatter (SP) and saturate (SA). We report the accuracies of each method w.r.t. each corruption on CIFAR-I 10-C, CIFAR-10K 100-C, and ImageNet-C in Table 14, Table ,and Table T6 Except for SODA-R and MA-SO using first- order gradients from the deployed model, SODA outperforms all baselines on almost all corruptions. On CIFAR-IO-C, SODA-R even outperforms MA-SO on all corruptions. On CIFAR-100-C, although the average accuracy of SODA-R is lower than that of MA-SO, SODA-R still outperforms MA-SO on 7 corruptions. On ImageNet-CC SODA improves the deployed model on most corruptions. SODA-R even outperforms or performs on par with MA-SO on corruptions.
Table 14: Accuracies of 19 coruuption CIFAR-10-C. For brevity, DA-PGI, DA-ZO0-Input, DA-PI, and DA-Direct are abbreviated as D-PG, D-Z-I, D-PL., and D-Di, respectively.
Deployed
DINE
BETA
D-PG
D-Z-I
D-PL
D-Di
SODA
SODA-R
MA-SO
GN
51.28
56.86
62.85
28.34
49.93
$2.18
48.80
73.18
85.82
84.09
ShN
56.02
58.44
64.75
29.52
53.99
56.63
54.59
74.52
86.23
85,40
IN
42.98
47.25
53.36
22.23
42.57
44.38
41.12
57.48
83.20
75.06
SpN
57.15
59.41
65.61
27.36
54.33
57.07
56.04
73.77
85.52
84.55
DB
88.16
88.14
86.94
16.39
84.64
88.67
85.94
90.97
91.79
90.98
GIB
49.21
53.31
58.38
17.48
46.65
49.75
44.68
66.24
77.08
76.51
MB
76.62
77.25
79.27
17.76
72.05
77.35
75.16
86.43
91.13
87.37
ZB
89.14
89.37
88.86
17.76
85.93
89.73
87.52
90.96
92.28
92.46
GaB 84.59
84.66
84.65
15.87
80.24
85.74
84.38
91.25
93.13
90.93
SW 78.06
78.03
77.42
36.16
75.39
78.62
75.53
83.83
88.92
85.94
FR 71.75
72.39
72.96
23.15
68.13
72.24
70.41
82.92
87.63
87.45
FG
70.58
71.84
73.60
11.56
63.08
71.56
71.71
82.72
86.29
82.82
BR 92.98
92.85
91.28
41.57
91.46
92.75
90.18
92.84
93.15
92.14
CT 86.72
86.74
84.64
15.06
67.58
87.95
87.15
92.67
93.89
92.07
ET 76.64
77.35
78.02
18.32
72.30
77.04
71.99
79.50
82.70
81.99
PX 52.12
58.46
64.50
27.95
50.91
52.35
49.55
87.12
90.26
89.29
JC
80.55
80.93
80.70
29.20
78.51
81.03
78.60
86.06
88.08
87.04
SP
77.66
77.11
77.80
30.20
76.13
77.86
75.69
83.06
88.87
85.73
SA
93.13
92.90
92.98
42.09
91.56
92.68
90.02
92.94
93.62
92.43
20

---

Table 15: Accuracies of 19 corruptions CIFAR-100-C, For brevity, DA-PGD, Inpuu,
DA-PL, and DA-Direct are abbreviated as D-PG, D-Z-I, D-PL, and D-Di, respectively.
Deployed
DINE
BETA
D-PG
D-Z-I
D-PL
D-Di
SODA
SODA-R
MA-SO
GN
19.21
20.17
20.89
5.31
13.95
19.13
16.73
40.88
53.59
57.25
ShN
22.13
23.02
24.23
5.28
16.20
21.87
19.49
41.91
55.16
58.60
IN
12.26
11.50
11.78
3.70
9.28
12.38
10.32
20.31
49.14
47.37
SpN
23.37
23.84
25.02
4.69
16.64
23.27
20.39
40.35
53.72
58.42
DB
60.39
57.75
56.31
3.24
44.99
60.00
55.37
67.22
68.03
68.92
GIB
17.74
17.81
18.58
3.01
12.61
17.22
12,90
29.92
41.97
49.66
MB
45.79
43.98
43.20
3.61
34.81
46.38
43.43
58.50
66.88
64.03
ZB 61.64
59.07
57.06
3.42
44.73
61.98
57.18
64.86
66.57
70.51
GaB 54.40
51.94
49.67
3.20
39.83
$5.09
51.40
68.53
SW
45.47
44.82
43.18
6.08
35.14
44.88
40.18
50.30
59.15
58.23
FR
39.77
39.65
39.52
3.49
27.85
39.77
37.11
52.10
57.82
60.97
FG
31.94
31.23
30.71
1,43
17.73
31.66
31.07
48.71
55.13
54.40
71.18
69.67
65.47
6.53
61.33
70.23
64.35
70.29
70.73
70.34
CT
49.10
46.10
43.35
1.23
25.49
51.56
48.43
71.71
73.03
69.91
ET
40.45
39.86
38.89
3.29
28.36
39.66
32.71
40.14
49.77
56.61
PX
27.77
27.87
29.36
4.24
29.23
27.72
24.85
55.63
62.06
66.37
JC
49.98
49.50
48.39
5.91
44.13
50.36
45.79
55.99
59.92
63.89
SP
44.18
43.80
42.47
5,03
35.22
44.56
39.20
61.85
61,83
SA
69.97
68.25
64.69
6.24
61.55
69.64
64.36
69.73
70.82
71.48
Table 16: Accuracies of 19 corruptions on ImageNet-C. For brevity, DA-PGD, DA-Z00-Input,
DA-PL, and DA-Direct abbreviated D-PG, D-Z--, D-PI., and D-Di, respectively. Note that DINE and BETA are excluded because their required initialization of the target models violates the setting of test-time adaptation.
Deployed
D-PG
D-Z-I
D-PL
D-Di
SODA
SODA-R
MA-SO
GN
8.98
6.72
10.52
10.38
9.18
26.72
40.99
34.85
ShN
11.48
9.39
6.14
12.93
28.46
43.24
40.46
IN
8.77
8.30
11.47
10.38
8.51
25.31
56.44
35.60
SpN
29.27
21.82
19.35
32.23
30.05
46.62
56.26
54.18
DB
21.99
1.29
2.42
21.73
17.47
24.52
31.43
43.98
GIB
7.35
2.30
0.74
6.78
1.69
7.84
0.62
38.46
MB
20.67
3.20
3.55
19.07
15.44
31.72
58.76
63.46
ZB
32.81
7.11
16.42
30.94
26.08
30.64
43.74
67.36
GaB
17.88
1.02
2.03
17.07
12.86
15.38
0.69
33.93
SW
34,92
1.33
14.55
33.98
31.15
51.58
59.39
63.52
36.88
1.63
7.57
37.70
36.27
54.42
58.40
FG
S7.72
0.91
32.85
58.07
57.87
60.92
60.45
73.17
BR
73.11
45.92
60.78
74.11
72.22
73.04
60.91
75.02
CT
70.92
0.13
50.83
75.02
75.18
74.72
61.71
69.21
ET
9.88
12.31
1.84
8.40
5.82
28.43
53.61
65.50
7.29
22.29
3.38
6.74
53,99
58.25
66.37
JC
47.32
43.11
24.12
49.15
47.63
51.08
52.52
59,22
SP
33.03
19.70
14.93
34.61
30.26
54.21
66.05
65.92
SA
65.53
42.17
50.42
67.06
65.87
69.25
65.87
72.50
21

---

test data
adapted data
defocus blur
glass blur
motion blur
zoom blur
gaussian blur
test data
adapted data
Gaussian noise
shot noise
impulse noise
speckle noise
saturate
test data
adapted data
snow
frost
fog
brightness
contrast
test data
adapted data
original data
elastic transform
pixlelate
jpeg compression
spatter
Figure 9: Examples of test data and adapted data using SODA for 19 corruptions in CIFAR-I0-C. The bottom-right data the original data in the CIFAR- 10 test dataset without corruption.
22


Reviews:
[{"review_id": "B6LQGJFkx6", "reviewer": "Reviewer_1NX6", "summary": "To better adapt models to test distributions without changing model parameters, this paper utilizes the strategy that trains a data adaptor which can adjust the test data to fit the deployed models. To avoid the potential corruption of data features caused by the data adaptor, the proposed method treats the test-time adaptation process as a semi-supervised learning process. Specifically, the test data points are split into two subsets, including a high confidence set to perform regular cross entropy minimization and a low confidence set (as unlabeled set) to perform mutual information maximization.", "questions": "Please refer to Weaknesses.", "limitations": "N/A.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper studies a realistic problem in test time adaptation, i.e., the unreliable nature of the pseudo-labels assigned to the test data. \n\n2. This paper proposes solves the low-quality issue of pseudo-labels by transforming the adaptation process as a semi-supervised learning process, in which the data adaptor model is less impacted by the mislabeled data points.\n\n3. The proposed method uses ZOO framework to estimate the gradients of the parameters and can efficiently the problem with a few queries.\n", "weaknesses": "1. The Pseudo-Label-Robust Data Adaptation module is the key contribution of this paper, but the design of this part is too simple. The problem here is actually noisy label learning problem and treating it as semi-supervised learning is a common strategy. \n\n2. The reliable pseudo-label selection process in Subsection 3.3 utilizes fixed threshold or ratio to select data points, which maybe not robust in real-world settings.\n\n3. The experiments on large scale datasets are not presented in the paper. Moreover, the parameter analyses in terms of $\\tau$ and $\\rho$ are needed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **Q1**: The Pseudo-Label-Robust Data Adaptation module is the key contribution of this paper, but the design of this part is too simple. The problem here is actually noisy label learning problem and treating it as semi-supervised learning is a common strategy.\n\n**A1**: We agree with your point that Pseudo-Label-Robust Data Adaptation is one of the key contributions, but we would like to highlight that the **challenges** have three folds.\n\n- **Inaccessible model parameters**. The parameters of target models are inaccessible, thus existing model adaptation methods may fail to promote the performance of target models. Accordingly, we propose to employ a data adaptor for target models.\n- **Infeasible gradients**. Calculating gradients using target models no longer holds in the scenario since model parameters are hidden. Therefore, we propose to employ zero-order optimization (ZOO) to approximate gradients for the update of the data adaptor.\n- **Noisy label**. The label of test samples is unknown, leading to biased loss values used in ZOO. To this end, we employ the commonly used pseudo-label strategy to perform robust data adaption. \n\nMoreover, inspired by your constructive comments, we further give **theoretical analysis** of the mentioned pseudo-label strategy under test-time adaptation scenarios, as shown in the general response.\n\n> **Q2:** The reliable pseudo-label selection process in Subsection 3.3 utilizes fixed threshold or ratio to select data points, which maybe not robust in real-world settings.\n\n**A2:** Thanks for your insightful comments. We agree with the point that fixed threshold and ratio may be suboptimal. As discussed in Appendix C.3, a good combination of threshold and ratio may be able to achieve better results. Adaptive noise ratio estimation like Gaussian Mixture Model (GMM)[1] might be of help to make SODA more robust. Meanwhile, we can see that SODA still outperforms baselines using fixed threshold and ratio.\n\n> **Q3:** The experiments on large scale datasets are not presented in the paper.\n\n**A3:** Thanks for your constructive suggestion, which motivates us to verify the efficacy of SODA on a large-scale dataset, i.e., ImageNet-C. The results are shown in the general response. We will add these results and discussion to our revised paper.\n\n\n> **Q4:** Moreover, the parameter analyses in terms of $\\tau$ and $\\rho$ are needed.\n\n**A4:** The hyper-parameter analyses are presented in Appendix C.3. In response to your kind reminder, we will highlight them on the main page.\n\n> References:\n> \n> [1] E. Arazo, et al, Unsupervised label noise modeling and loss correction, ICML, 2019.\n"}, {"review_id": "1tJX0rbtJ4", "reviewer": "Reviewer_SVqY", "summary": "This paper proposes usage of zeroth order optimization (ZOO) for test-time adaptation (TTA) to ease several practical issues regarding accessing model parameters during TTA. Since the ZOO with pseudo label, which is a standard method in TTA, might cause the unreliable gradient, the paper proposes a sample selection method using the confidence of the prediction and class balance, and use only reliable sample to compute pseudo-label loss. The unreliable sample is used to compute another unsupervised loss to facilitate better adaptation. They show its effectiveness, CIFAR10-C and CIFAR100-C. No theory is provided.", "questions": "See weakness section", "limitations": "None.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is generally well written and easy to follow. \n2. The usage of zeroth-order optimization in TTA is well motivated and interesting new problem. \n3. The proposed sample selection approach based on the confidence and class balance seems not to be revolutionary but sensible.", "weaknesses": "1. The experiment is limited to the CIFAR10-C and CIFAR100-C. As usual, I recommend adding experiments on ImageNet-C and some domain adaptation datasets. \n\n2. The technical novelty is not high. Besides, I found that the effectiveness of the selection method proposed in this paper is not fully validated. For example, the paper does not provide ablation about the sensitivity about the threshold parameter. I'm also curious why we should not compute information maximization loss for reliable samples. Besides, the necessity of information maximization loss is not experimentally validated. \n\n3. The selection of the information maximization loss is not well described. Why did you choose the specific loss function? \n\n4. Several details are unclear for me. Including, \n \n- What the difference between SODA-R and SODA-FO? \n- I'm confused by the table 5, since it says that SODA-O as a variant of SODA under *online* settings but seems to repeat the optimization multiple epochs. Do you repeat the optimization after you reach all test dataset? In the case, I have to say that it is not usual online setup. \n\n5. No theoretical results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **Q1:** The experiment is limited to the CIFAR10-C and CIFAR100-C. As usual, I recommend adding experiments on ImageNet-C and some domain adaptation datasets.\n\n**A1:** Thanks for your constructive suggestions. More experimental results on ImageNet-C and challenging Office-Home tasks are shown in the attached file in general response.\n\n> **Q2:** The technical novelty is not high.\n\n**A2:** We would like to highlight our main novelty and contributions in the following three folds:\n- **ZOO for test-time data adaptation**: We tackle a challenging and realistic setting where the parameters of the deployed model are inaccessible and unmodifiable. zeroth-order optimization and data adaptation are proposed to solve this problem.\n- **Label noise in ZOO**: We analyze the effect of label noise in ZOO, and point out that the noisy pseudo-labels can cause biased gradient estimation in ZOO, leading to limited performance of test-time data adaptor.\n- **New methods for robust test-time data adaptation**: Based on our analysis, we propose SODA to robustly train the test-time data adaptor. SODA separates the pseudo-labels into reliable and unreliable sets and performs semi-supervised learning using cross-entropy loss and mutual information maximization.\n\n> **Q3:** Besides, I found that the effectiveness of the selection method proposed in this paper is not fully validated. For example, the paper does not provide ablation about the sensitivity about the threshold parameter. \n\n**A3:** The hyper-parameter analysis is presented in Appendix C.3. In response to your kind reminder, we will highlight it in the main paper.\n\n> **Q4:** I'm also curious why we should not compute information maximization loss for reliable samples.\n\n**A4:** Information maximization works by making the model prediction more certain while keeping diversity in the global structure. The predictions of data samples with reliable labels already have high confidence, thus information maximization is not needed for those samples.\n\n> **Q5:** Besides, the necessity of information maximization loss is not experimentally validated.\n\n**A5:** The baseline DA-PL in our main experiments shows the effectiveness of information maximization. DA-PL only uses the pseudo-labels to train the data adaptor and makes trivial improvements. Compared to DA-PL, our proposed SODA improves the deployed model to a large extent by separating the dataset into a reliable set supervised trained by reliable pseudo-labels and an unreliable set unsupervised trained by information maximization.\n\n> **Q6:** The selection of the information maximization loss is not well described. Why did you choose the specific loss function?\n\n**A6:** Thanks for your helpful comments. Because of the high error rate, data samples with unreliable pseudo-labels may be misclassified into classes with large amounts of samples, and hard to separate them. Following previous works [1][2], One useful way to circumvent this problem is to encourage diversity among predictions of each data sample. Information maximization is a widely-used unsupervised loss that can encourage both global diversity and local certainty of model predictions. In response to your kind reminder, we will add this explanation to our revised paper.\n\n> **Q7:** Several details are unclear for me. Including, What the difference between SODA-R and SODA-FO\n\n**A7:** SODA-R and SODA-FO are both relaxed baselines assuming that gradient computation is allowed from the deployed model while the parameters of the deployed model are still not modifiable. They both use first-order optimization to compute gradients for the training of the data adaptor. As a comparison baseline, SODA-FO keeps everything the same as SODA except the usage of FOO, to show the effect of ZOO in the training of the data adaptor. Based on SODA-FO, SODA-R adopts deeper network architecture of the data adaptor, Adam optimizer, perturbation regularization, and dropout strategy to show better results which can be achieved by SODA under relaxed settings. A more detailed discussion is presented in Appendix C.1. Thanks for your comments, we will put a more detailed explanation in our revised paper.\n\n> **Q8:** I'm confused by the table 5, since it says that SODA-O as a variant of SODA under online settings but seems to repeat the optimization multiple epochs. Do you repeat the optimization after you reach all test dataset? In the case, I have to say that it is not usual online setup.\n\n**A8:** Sorry about the confusion. The optimization in SODA-O is not repeated after reaching the entire test dataset but only repeats for the current test data batch and the cached queue. During the adaptation of the current test data batch, the previous data batches are no longer available except for those saved in the queue. After reaching the entire test dataset, the whole adaptation process ends. Thanks for your comments, we will put a more detailed explanation in our revised paper.\n\n> **Q9:** No theoretical results.\n\n**A9:** Thanks for your instructive comments. We provide theoretical analysis about pseudo-label-robust training and ZOO in the general response.\n\n> References:\n> \n> [1] Jian Liang, et al. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. ICML, 2020.\n> \n> [2] Jian Liang, et al. Dine: Domain adaptation from single and multiple black-box predictors. CVPR, 2022\n"}, {"review_id": "vTY0pwx9lD", "reviewer": "Reviewer_xmoq", "summary": "In this work, the authors aim to adapt unlabelled test data to a deployed model without access to its parameters and inner structures during the testing process. Specifically, the authors utilize a data adaptor during testing to map test data into the deployed model, which gradients are estimated via ZOO. Experiments are conducted on CIFAR-10C and CIFAR-100C to verify the effectiveness of the proposed method.", "questions": "See the Weaknesses.", "limitations": "The authors provide the limitations of their work. ", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. This paper is well-written and easy-to-follow. \n\n2. The code is available with the submission which improves the reproducibility index of the paper.\n", "weaknesses": "1. Experiments are not convincing. The authors only use CIFAR-10C and CIFAR-100C to verify the effectiveness of their algorithm. I suggest more datasets with various types of distribution shift should be included such as Office-31, Office-Home, PAC, etc.\n\n2. The necessity of ZOO is not clear. I can understand parameters of the deployed model are inaccessible, but why the data adaptor you generate during testing is still a black box? If the parameters of the data adaptor are known, why do not you simply fix the parameters of the deployed model as constants, and use FOO to calculate gradients of the data adaptor?\n\n3. More baselines should be considered. There already exist some TTA algorithms that do not requiring access to parameters of the deployed model, such as T3A [1].\n\n[1] Iwasawa Y, Matsuo Y. Test-time classifier adjustment module for model-agnostic domain generalization[J]. Advances in Neural Information Processing Systems, 2021, 34: 2427-2440.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **Q1:** Experiments are not convincing. The authors only use CIFAR-10C and CIFAR-100C to verify the effectiveness of their algorithm. I suggest more datasets with various types of distribution shift should be included such as Office-31, Office-Home, PAC, etc.\n\n**A1:** Thanks for your instructive suggestion, we further conduct experiments on ImageNet-C and challenging Office-Home domain adaptation tasks to illustrate the efficacy of our proposed SODA framework. The results are shown in the attached file in the general response.\n\n> **Q2:** The necessity of ZOO is not clear. I can understand parameters of the deployed model are inaccessible, but why the data adaptor you generate during testing is still a black box? If the parameters of the data adaptor are known, why do not you simply fix the parameters of the deployed model as constants, and use FOO to calculate gradients of the data adaptor?\n\n**A2:** Sorry about the confusion. In our work, the data adaptor is not a black box, and its parameters can be accessed and modified. In our settings, the parameters of the deployed model are hidden, so **backward propagation through the deployed model is not allowed**, leading to infeasible gradient computation. Hence, zeroth-order optimization is used to circumvent this problem and estimate gradients w.r.t. parameters of the data adaptor for training of the data adaptor.\n\n> **Q3:** More baselines should be considered. There already exist some TTA algorithms that do not require access to parameters of the deployed model, such as T3A [1].\n\n**A3:** Thanks for your constructive suggestion. We also expect more proposed works dealing with the same settings as we do, however, most of the existing works which do not modify the model parameters require access to the extracted features. For example, T3A[1] splits the pre-trained model into a feature extractor and classifier, and uses the features extracted from the feature extractor to form the support set. Compared to their settings, our settings forbid access to features that are stricter and more practical due to intellectual property protection, misuse prevention, privacy concerns in healthcare and finance, etc.\n\n> References:\n> \n> [1] Iwasawa Y, Matsuo Y. Test-time classifier adjustment module for model-agnostic domain generalization[J]. Advances in Neural Information Processing Systems, 2021, 34: 2427-2440.\n"}, {"review_id": "k3JUGLZQga", "reviewer": "Reviewer_H74J", "summary": "This paper proposes SODA, a test-time data adaptor with the black-box source model leveraging Zeroth-Order Optimization for the adaptor, which involves a random perturbation on the adaptor model's parameters. It also considers the scenario, namely SODA-R when the gradient information is available and online setting SODA-O. The proposed method outperforms existing benchmarks, including BETA and DINE. The experiment is comprehensive with CIFAR10-C/CIFAR100-C. However, having more benchmark dataset experiments is better for achieving better confidence with the proposed framework for practical application.", "questions": "1. Do we need to apply data augmentations in the SODA framework? I guess not, but data augmentation information, including DINE and BETA, seems missing. In another sense, data augmentation conflicts (like Dropouts) with applying the model perturbation in mutual information calculation - Eq. $(7)$. So if we don't have to apply Data Augmentation, what is the primary source of the outperformance? I would like to see some convincing evidence of the hypothesis.\n2. Aligning with the first question, what is the source of the randomness in calculating Eq. $(7)$? Specifically, the perturbation on $\\theta$? or data $x_i$? I would like to have confirmation from the authors.", "limitations": "The main limitation of this paper is the scope of the experiments, although an extensive study in CIFAR10-C/CIFAR100-C. There are many other benchmarks in the domain-adaptation, such as Office-31, VISDA, Office-Home, etc. It would be more convincing if the authors could demonstrate the out-performance with a fixed set of hyperparameters.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Test-time adaptor problem is well-motivated, and leveraging the adaptor could potentially save the training cost in practice.\nUnder the limited available information scenario, the authors claim that they successfully tackle the problem better than other benchmarks, such as BETA and DINE. I value the simplicity of the approach only with two components; 1) mutual information maximization and 2) cross-entropy loss with pseudo labels by achieving a better performance. As a black-box scenario, overcoming the lack of gradient is another main ingredient of this paper, and it outperforms other benchmarks in CIFAR10-C/CIFAR100-C. Application of Zeroth-Order Optimization is another key ingredient, but the simplicity is better appealing to me.", "weaknesses": "1. I would like to understand the relationship between the model perturbation and the data augmentation in the proposed framework. See the more detailed question in the Questions section.\n2. A good combination of the choice $\\sigma, \\alpha, \\tau$ seems to be critical. I see this discussion at C3 in the Appendix, but I suggest that the authors make this more explicit in the main paper. If those parameters are dataset/or task-dependent, it's not yet applicable in practice.\n3. Why $\\sigma=0.5$ generally achieve a better performance? And a better performance with $\\alpha=0.0001$ implies that the mutual information term is the most critical. A further ablation study of each loss term would be necessary.\n\nMinor Comments:\n- L61: SOTA $\\to$ SODA?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> **W1:** I would like to understand the relationship between the model perturbation and the data augmentation in the proposed framework.\n\n**AW1:** There are **two kinds of “perturbation”** in our proposed work:\n- **Parameter perturbation** used in gradient estimation of zeroth-order optimization (ZOO) [1]. It is used to overcome the inaccessible gradient problem. \n- **Data adaptation** achieved by the data adaptor.\n\n**Data adaptation is different from data augmentation** in two points:\n\n- Working scheme: data adaptation generates perturbations using the network and adds them to the original data samples, while data augmentation performs pre-defined visual transformations to data samples.\n- Purpose: the perturbations generated in data adaptation are used to adapt test data to the deployed model, while data augmentation is usually to reduce overfitting in neural network training.\n\n> **Q1:** Do we need to apply data augmentations in the SODA framework? I guess not, but data augmentation information, including DINE and BETA, seems missing.\n\n**AQ1:** Data augmentation is not considered in SODA and DINE, but is used in BETA. \n\n> **Q2:** Aligning with the first question, what is the source of the randomness in calculating Eq. (7)? Specifically, the perturbation on $\\theta$ or data $x_i$? I would like to have confirmation from the authors.\n> \n> **Q3:** In another sense, data augmentation conflicts (like Dropouts) with applying the model perturbation in mutual information calculation - Eq. (7).\n\n**AQ2&3:** First, we would like to explain the working scheme of mutual information maximization (IM). IM [2] has two terms:\n- Conditional entropy which encourages the model prediction to be more certain and form tighter clusters for each class.\n- Marginal entropy which encourages the model prediction to be more diverse and form more separated clusters among classes.\n\nIn the calculation of IM, the randomness comes from the distribution of the test data and the uncertainty of the model prediction, not from perturbations on either $\\boldsymbol{\\theta}$ or $\\mathbf{x}_i$. Data augmentation is not conflicted with the calculation of IM in Eq.(7).\n\n> **Q4:** So if we don't have to apply Data Augmentation, what is the primary source of the outperformance? I would like to see some convincing evidence of the hypothesis.\n\n**AQ4:** We agree that data augmentation is a powerful method to improve the performance of models, but the key idea of SODA is data adaptation instead of data augmentation. The effectiveness of SODA comes from the training of the data adaptor to generate adapted data. The training objective consists of two components:\n- Supervised training with reliable pseudo-labels to alleviate the data corruption problem caused by unreliable pseudo-labels.\n- Unsupervised training of data samples with unreliable pseudo-labels to encourage the model prediction on those data samples to be certain and diverse.\n\n> **W2:** A good combination of the choice $\\sigma$, $\\alpha$, $\\tau$ seems to be critical. I see this discussion at C3 in the Appendix, but I suggest that the authors make this more explicit in the main paper.\n\n**AW2:** Thanks for your constructive suggestions. We agree with the point that choosing a good combination of hyper-parameters may be suboptimal. But as discussed in Appendix C3, our proposed SODA is robust to most combinations. With your kind reminder, we will put this discussion on our main page. \n\n> **W3:** If those parameters are dataset/or task-dependent, it's not yet applicable in practice.\n\n**AW3:** Thanks for your insightful comments. We also agree that the need of choosing good hyper-parameters may hinder the practicability of our work. But for CIFAR-10-C and CIFAR-100-C datasets used in our main experiments, accuracies before adaptation range from ~10% to ~90% for different corruptions. Using a fixed threshold and ratio, our proposed SODA improves the deployed model for almost all corruptions, showing that SODA can handle distribution shifts to various extents, and is not specifically dataset dependent.\n\n> **W4:** Why $\\sigma$ = 0.5 generally achieve a better performance?\n\n**AW4:** The hyper-parameter analysis experiments are conducted on CIFAR-10-C Gaussian noise corruption. Before adaptation, the initial accuracy is 51.28%. Since $\\rho$ ($\\sigma$ in your question) should ideally be the noise ratio in the pseudo-labels, i.e. the error rate of the deployed model before adaptation, setting $\\rho=0.5$ is expected to have better results. However, in our main experiments, we do not elaborately choose different $\\rho$ for different kinds of corruption with different initial accuracies. SODA with a fixed $\\rho$ has already improved the deployed model to a large extent.\n\n> **W5:** And a better performance with $\\alpha$ = 0.0001 implies that the mutual information term is the most critical. A further ablation study of each loss term would be necessary.\n\n**AW5:** Thanks for your instructive suggestion. Both loss terms in SODA are useful. Besides the discussion in Appendix C3, we also show the effectiveness of each loss term in the Office-Home Art->Clipart task:\n\n|Office-Home|Deployed|Pseudo-label only|IM only|SODA|\n|-|-|-|-|-|\n|Art->Clipart|44.47%|44.99%|45.36%|46.53%\n\n> **Minor Comments**: L61: SOTA  SODA?\n\n**A:** Thanks for pointing out this typo, it should be SODA, we will fix it in our revised paper.\n\n**Answer for limitations:** With your constructive suggestion, we provide more experimental results in the general response. We also agree that using adaptive thresholds might be a promising improvement in the future.\n\n> References:\n> \n> [1] Sijia Liu, et al. A primer on zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications. IEEE Signal Processing Magazine, 2020.\n> \n> [2] Shi, Yuan, and Fei Sha. Information-theoretical learning of discriminative clusters for unsupervised domain adaptation. ICML 2012.\n\n"}]

Author rebuttal:
We sincerely appreciate all reviewers for taking the time and effort to review our paper and provide valuable feedback. We would like to thank reviewers for their recognition of our work: 1) our problem is **realistic** (#1NX6), **well-motivated** and **interesting new** (#SVqY and #H74J); 2) our method is **efficient** (#1NX6), **effective** (#SVqY, #xmoq, #H74J), **simple** (#H74J) and **sensible** (#SVqY); 3) our paper is **well-written** and **easy-to-follow** (#SVqY and #xmoq).

Besides the response to each reviewer, we would like to provide more **experimental results** on ImageNet-C and challenging Office-Home domain adaptation tasks as constructively suggested by reviewers in the attached PDF file.

Furthermore, we would like to provide more **theoretical analysis** here to show that our proposed pseudo-label-robust training strategy can tighten the upper bound of the expected gradient estimation error in zeroth-order optimization.
> 
> Given a data adaptor $\mathbf{G}$ with parameter $\boldsymbol{\theta}$, a deployed model $\mathbf{M}$ and a test dataset $\mathbf{X} = \\{\mathbf{x}_1,...,\mathbf{x}_n\\}$, denote the adapted data sample as ${\mathbf{x}_i^{\boldsymbol{\theta}}}$, the true label of $\mathbf{x}_i$ as $\mathbf{y}_i$, and $\hat{\mathbf{p}}_i^{\boldsymbol{\theta}}=\mathbf{M}\circ \mathbf{G}(\mathbf{x}_i;\boldsymbol{\theta})$.
> 
> According to [1], minimizing the cross entropy loss $\mathcal{L}_{\rm ce}(\mathbf{y}_i, \hat{\mathbf{p}}_i^{\boldsymbol{\theta}})$ is equivalent to maximizing the mutual information $\mathcal{L} _{\rm im}(\mathbf{x}_i^{\boldsymbol{\theta}})$. 
> 
> From derivation in Appendix A, with pseudo-label $\hat{\mathbf{y}}_i = \mathbf{y}_i + \boldsymbol{\sigma}_i$, the KL divergence loss at test data point $\mathbf{x}_i$ is:
> $$\mathcal{L}_i = -H(\mathbf{y}_i+\boldsymbol{\sigma}_i)+\mathcal{L} _{\rm ce}(\mathbf{y}_i, \hat{\mathbf{p}}_i^{\boldsymbol{\theta}}) - \boldsymbol{\sigma}_i \log \hat{\mathbf{p}}_i^{\boldsymbol{\theta}}.$$
> 
> Denoting $h(\mathbf{x}_i) = -\boldsymbol{\sigma}_i \log \hat{\mathbf{p}}_i^{\boldsymbol{\theta}}$, the gradient of the KL divergence loss is:
> $$\nabla _{\boldsymbol{\theta}}\mathcal{L}_i = \nabla _{\boldsymbol{\theta}}\mathcal{L} _{\rm ce} + \nabla _{\boldsymbol{\theta}}h.$$
>
> Then, in gradient estimation of ZOO, the estimated gradient of the KL divergence loss is:
> $$\widehat{\nabla} _{\boldsymbol{\theta}}{\check{\mathcal{L}} _i} = \widehat{\nabla} _{\boldsymbol{\theta}}\mathcal{L} _{\rm ce} + \widehat{\nabla} _{\boldsymbol{\theta}}h.$$
> 
> Hence, before applying pseudo-label-robust data adaptation, the upper bound of expected gradient estimation error is:
> $$\mathbb{E}[\parallel \widehat{\nabla} _{\boldsymbol{\theta}}{\check{\mathcal{L}} _i} - \nabla _{\boldsymbol{\theta}}\mathcal{L} _i \parallel_2] \leq \mathbb{E}[\parallel \widehat{\nabla} _{\boldsymbol{\theta}}{\check{\mathcal{L}} _{\rm ce}} - \nabla _{\boldsymbol{\theta}}\mathcal{L} _{\rm ce} \parallel_2] + \mathbb{E}[\parallel \widehat{\nabla} _{\boldsymbol{\theta}}h - \nabla _{\boldsymbol{\theta}}h \parallel_2].$$
> 
> In SODA, by separating $\mathbf{X}$ into reliable set $\mathbf{X}_r$ learned by cross-entropy loss with pseudo-labels, and unreliable set $\mathbf{X}_u$ learned by mutual information loss, the expected gradient estimation error on the whole dataset is: 
$$
\begin{aligned}
& \mathbb{E} _{\mathbf{X}}[\mathbb{E}[\parallel \hat{\nabla} _{\boldsymbol{\theta}}{\check{\mathcal{L}} _i} - \nabla _{\boldsymbol{\theta}}\mathcal{L} _i \parallel _2]] \\\\ & = \mathbb{E} _{\mathbf{X} _r}[\mathbb{E}[\parallel \hat{\nabla} _{\boldsymbol{\theta}}{\check{\mathcal{L}} _i} - \nabla _{\boldsymbol{\theta}}\mathcal{L} _i \parallel_2]] + \mathbb{E} _{\mathbf{X} _u}[\mathbb{E}[\parallel \hat{\nabla} _{\boldsymbol{\theta}}{\check{\mathcal{L}} _i} - \nabla _{\boldsymbol{\theta}}\mathcal{L} _i \parallel _2]] \\\\ & = \mathbb{E} _{\mathbf{X} _r}[\mathbb{E}[\parallel \hat{\nabla} _{\boldsymbol{\theta}}\mathcal{L} _{\rm ce} - \nabla _{\boldsymbol{\theta}}\mathcal{L} _{\rm ce} + \hat{\nabla} _{\boldsymbol{\theta}}h - \nabla _{\boldsymbol{\theta}}h \parallel _2]] + \mathbb{E} _{\mathbf{X} _u}[\mathbb{E}[\parallel \hat{\nabla} _{\boldsymbol{\theta}}{\mathcal{L} _{\rm im}} - \nabla _{\boldsymbol{\theta}}\mathcal{L} _{\rm im} \parallel _2]] \\\\
& \leq \mathbb{E} _{\mathbf{X} _r}[\mathbb{E}[\parallel \hat{\nabla} _{\boldsymbol{\theta}}\mathcal{L} _{\rm ce} - \nabla _{\boldsymbol{\theta}}\mathcal{L} _{\rm ce} \parallel _2] + \mathbb{E}[\hat{\nabla} _{\boldsymbol{\theta}}h - \nabla _{\boldsymbol{\theta}}h \parallel _2]] + \mathbb{E} _{\mathbf{X} _u}[\mathbb{E}[\parallel \hat{\nabla} _{\boldsymbol{\theta}}{\mathcal{L} _{\rm ce}} - \nabla _{\boldsymbol{\theta}}\mathcal{L} _{\rm ce} \parallel _2]] \\\\
& \leq \mathbb{E} _{\mathbf{X}}[\mathbb{E}[\parallel \hat{\nabla} _{\boldsymbol{\theta}}\mathcal{L} _{\rm ce} - \nabla _{\boldsymbol{\theta}}\mathcal{L} _{\rm ce} \parallel _2] + \mathbb{E}[\hat{\nabla} _{\boldsymbol{\theta}}h - \nabla _{\boldsymbol{\theta}}h \parallel _2]].
\end{aligned}$$
>
> Thus, the upper bound of gradient estimation error is tightened after applying pseudo-label-robust data adaptation.

> Reference:
> 
> [1] Boudiaf, Malik, et al. A unifying mutual information view of metric learning: cross-entropy vs. pairwise losses. ECCV, 2020.


Meta-review: The proposed work on test-time data adapators for learning settings without access to model parameters is considered novel enough and initial concerns on theoretical and experimental justification on more realistic data have been sufficiently addressed in the discussion phase. Although the paper has no strong support, none of the reviewers is against acceptance. The AC has read the paper and after discussion with the SAC concludes that with the promised changes added to the camera-ready the work is acceptable. Adding more experimental results on non-synthetic datasets as commonly used in the test-time adaptation literature would strengthen the work even further.
Decision: Accept (poster)